Recognizing Facial Expressions in Videos Using a Facial Action Analysis-Synthesis Scheme

In this paper, we propose a novel approach for facial expression analysis and recognition. The proposed approach relies on tracked facial actions provided by an appearance-based 3D face tracker. For each universal expression, a dynamical model for facial actions given by an auto-regressive process is learned from training data. We classify a given image in an unseen video into one of the universal facial expression categories using an analysis-synthesis scheme. This scheme uses all models and select the one that provides the most consistent synthesized spatio-temporal facial actions. The dynamical models can be utilized in the tasks of synthesis and prediction. Experiments using unseen videos demonstrated the effectiveness of the developed method.