A tutorial introduction to estimation and filtering

In this tutorial paper the basic principles of least squares estimation are introduced and applied to the solution of some filtering, prediction, and smoothing problems involving stochastic linear dynamic systems. In particular, the paper includes derivations of the discrete-time and continuous-time Kalman filters and their prediction and smoothing counterparts, with remarks on the modifications that are necessary if the noise processes are colored and correlated. The examination of these state estimation problems is preceded by a derivation of both the unconstrained and the linear least squares estimator of one random vector in terms of another, and an examination of the properties of each, with particular attention to the case of jointly Gaussian vectors. The paper concludes with a discussion of the duality between least squares estimation problems and least squares optimal control problems.