Individual 3D face synthesis based on orthogonal photos and speech-driven facial animation

In the paper, a methodology for individual face synthesis using given orthogonal photos is proposed. An integrated speech-driven facial animation system is presented. Firstly, in order to capture a given subject's personal facial configuration, a novel coarse-to-fine strategy based on facial texture and deformable template is proposed to localize some facial feature points in the image of frontal view. The corresponding feature points in the profile are extracted by using the polygonal approximation. Secondly, all these feature points are aligned to fit the generic 3D face model to a specialized one to reflect the given person's facial configuration. Then a multi-direction texture-mapping technique is presented to synthesize a lifelike personal face. Finally, muscle-based expression and lip-motion models are built up. All the above technologies are integrated into a speech-driven face animation system. We are aiming at a MPEG-4 compatible video-driven face animation system.