3D facial tracking from corrupted movie sequences

In this paper we perform 3D face tracking on corrupted video sequences. We use a deformable model, combined with a predictive filter to recover both the rigid transformations and the values of the parameters that describe the evolution of the facial expressions over time. To be robust, predictive filters need a good observation of the system's state. We describe a new method to measure, at each moment in time, the correct distribution of an observation of the parameters of a high-dimensional deformable model. This method is based on bounding the confidence regions of the 2D image displacements with affine forms, and propagating them into parameter space. Using Lindeberg's theorem, we measure a good Gaussian approximation of the parameters in a manner that avoids many of the traditional assumptions about the observations' distributions. We demonstrate in experiments on sequences with compression artifacts, and poor-quality video sequences of Lauren Bacall and Humphrey Bogart from the 1950s, that, without any learning involved, our method is sufficiently robust to extract information from degraded image sequences. In addition, we provide ground truth validation.