Real-Time 3D Face and Facial Action Tracking Using Extended 2D+3D AAMs

In this work, we address the problem of tracking three dimensional (3D) faces and facial actions in video sequences. The main contributions of the paper are as follows. First, we develop an extended 2D+3D Active Appearance Models (AAM) based 3D face and facial action tracking framework using 2D view-based AAMs and a modified 3D face model. Second, we develop a robust shape initialization method based on local feature matching to track fast face motion. Experiments evaluating the effectiveness of the proposed algorithm are reported.