Fast Landmark Localization With 3D Component Reconstruction and CNN for Cross-Pose Recognition

Two approaches are proposed for cross-pose face recognition, one is built on the handcrafted features extracted from the 3D reconstruction of facial components and the other is built on the learned features from a deep convolutional neural network (CNN). As both approaches rely on facial landmarks for alignment across large poses, we propose the Fast Hierarchical Model (FHM) for locating cross-pose facial landmarks in real time. Unlike most 3D approaches that consider holistic faces, the first proposed approach considers 3D facial components. It segments each 2D face in the gallery into components, reconstructs the 3D surface for each component, and recognizes a query face by component features. The core part of the CNN-based approach is a modified VGG network. We study the performance with different settings on the training set, including the synthesized data from 3D reconstruction, the real-life data from an in-the-wild database, and both types of data combined. The two recognition approaches and the FHM are evaluated in extensive experiments and compared with state-of-the-art methods to demonstrate their efficacy.