Real-time 3-D face tracking and modeling from awebcam

We first infer a 3-D face model from a single frontal image using automatically extracted 2-D landmarks and deforming a generic 3-D model. Then, for any input image, we extract feature points and track them in 2-D. Given these correspondences, sometimes noisy and incorrect, we robustly estimate the 3-D head pose using PnP and a RANSAC process. As the head moves, we dynamically add new feature points to handle a large range of poses. When the tracker gets lost, due to motion blur or occlusions, the system re-initializes by matching feature points to the reference frontal image feature points. Our system runs in real-time (>;15Hz) on a standard CPU with a GPU card. We present results on stored video and will present a live demo, showing excellent tracking under large motion, fast movement, occlusion and facial expression variations. We also show comparative results with the ground truth BU head tracking dataset.