Shape from recognition and learning: recovery of 3-D face shapes

In this paper a novel framework for the recovery of 3D surfaces of faces from single images is developed. The underlying principle is shape from recognition, i.e. the idea that pre-recognizing face parts can constrain the space of possible solutions to the image irradiance equation, thus allowing robust recovery of the 3D structure of a specific part. Shape recovery of the recognized part is based on specialized backpropagation based neural networks, each of which is employed in the recovery of a particular face part. Representation using principal components allows to efficiently encode classes of objects such as nose, lips, etc. The specialized networks are designed and trained to map the principal component coefficients of the shading images to another set of principal component coefficients that represent the corresponding 3D surface shapes. A method for integrating recovered 3D surface regions by minimizing the sum squared error in overlapping areas is also derived. Quantitative analysis of the reconstruction of the surface parts show relatively small errors indicating that the method is robust and accurate. The recovery of a complete face is performed by minimal squared error merging efface parts.