Recovery of 3-D face structure using recognition

A novel framework for the recovery of 3D surfaces of faces from single images is developed in this work. The underlying principle is shape from recognition, i.e. the idea that pre-recognizing face parts can constrain the space of possible solutions to the image irradiance equation, thus allowing robust recovery of the 3D structure of a specific part. Generic face parts are localized using expansion matching filters. Specialized backpropagation based neural networks are then employed in the recovery of the recognized and localized face part. This circumvents directly solving the image irradiance equation. Instead, the relationships between variations in face structure and appearance under varying pose and illumination is learned. Representation using principal components allows to efficiently encode classes of objects such as nose, lips, etc. for association with the neural networks. Quantitative analysis of the reconstruction of the surface parts show relatively small errors, indicating that this system can accurately recover 3D face structure from single images invariant to pose and illumination.