Real-time estimation of long-term 3-D motion parameters for SNHC face animation and model-based coding applications

We present two recursive methods for the real-time estimation of long-term three-dimensional (3-D) motion parameters from monocular image sequences suitable for synthetic/natural hybrid coding face animation and model-based coding applications. Based on feature point extractions in energy frame, the 3-D motion parameters of a human face are estimated with a predictive approach. The first method uses a recursive linear least squares approach and the second employs a nonlinear extended Kalman filter, which does not rely on a linearized model of the face motion. Both methods perform a prediction and correction loop at every time step. Compared to other methods described in the literature, the recursive and predictive structure of the proposed estimation process solves the problem of error accumulation in long-term motion estimation. This makes the estimation stable and consistent over long periods. Experimental results are presented for synthetic data and real image sequences, which demonstrate the performance of the estimation methods and compare the two approaches.