LS-SIFT: Enhancing the Robustness of SIFT During Pose-Invariant Face Recognition by Learning Facial Landmark Specific Mappings

The proper functioning of many real-world applications in biometrics and surveillance depends on the robustness of face recognition systems against pose, and illumination variations. In this work, we employ ensemble systems in conjunction with local descriptors to address pose-invariant face recognition (PIFR). Facial landmarks are detected during the first step with a two fold usage. The landmark locations are employed to perform head pose classification (HPC). HPC allows to select only the visible landmarks for further processing. Then, local descriptors are extracted from the selected landmarks within a face image. We are proposing a novel learned descriptor (LS-SIFT) to overcome the robustness limitations of SIFT against large viewpoint variability during face recognition. Second, the extracted descriptors are used to train the base learners comprising an ensemble system for each subject in a face database (one ensemble per subject, one base learner per landmark). A novel GMM-based base learner model, named Mahalanobis Similarity (MS), is introduced in this work. Finally, face recognition is performed based on the ensemble systems’ outputs from all the subjects in a face database. During the experimental trials, SIFT and LS-SIFT are employed individually for local feature extraction, whereas GMM and MS are used to build the ensemble systems, in an independent manner, for further comparison. The whole PIFR system is tested on CMU-PIE, Multi-PIE, and FERET databases, outperforming most of the state-of-the-art works regarding images with pose angles in the range of
$\pm 90^{o}$
.