3-D Generic Elastic Models for Fast and Texture Preserving 2-D Novel Pose Synthesis

This paper provides an in-depth analysis on face shape alignment for pose insensitive face recognition. The dissimilarity between two face images can be modeled as the difference in intensity between these two images, obtained by warping these faces onto the same shape. In order to achieve this, we must first align both face images independently to obtain a sparse 2-D shape representation. We achieve this by using a Combination of ASMs and AAMs (CASAAMs). We then exchange these two shapes and obtain new intensity (texture) faces based on these exchanged shapes. This allows us to align the two faces with increased pixel-level correspondence while simultaneously achieving a certain degree of pose correction. In order to account for large pose variation, it becomes necessary to model the underlying 3-D face structure for the synthesis of novel 2-D poses. However, in many real-world scenarios, only a single image of the subject is provided and acquisition of the 3-D model is not always feasible. To tackle this common real-world scenario, we propose a novel approach for modeling faces, called 3D Generic Elastic Model (3D-GEM), which can be deformed from a single 2-D image. Our analysis shows that 3-D depth information of human faces does not dramatically change across people, indicating that precise depth information of a person is not needed to generate useful novel 2-D poses. This is a significant discovery that makes our method feasible. We thus demonstrate that our 3-D face model can be efficiently produced by using a generic depth model which can be elastically deformed based on input facial features. This face model can then be rotated in 3-D in order to synthesize any arbitrary 2-D facial pose. Experimental results show that 3-D faces modeled by our proposed work effectively handle large 3-D pose changes in face alignment and they can be used for achieving pose tolerant face recognition. We also provide comparative results of face synthesis obtained by an actual 3-D face scanner and our approach, showing our proposed modeling approach is both effective and efficient.