Robust 3D face recognition in the presence of realistic occlusions

Facial occlusions pose significant problems for automatic face recognition systems. In this work, we propose a novel occlusion-resistant three-dimensional (3D) facial identification system. We show that, under extreme occlusions due to hair, hands, and eyeglasses, typical 3D face recognition systems exhibit poor performance. In order to deal with occlusions, our proposed system employs occlusion-resistant registration, occlusion detection, and regional classifiers. A two-step registration module first detects the nose region on the curvedness-weighted convex shape index map, and then performs fine alignment using nose-based Iterative Closest Point (ICP) algorithm. Occluded areas are determined automatically via a generic face model. After non-facial parts introduced by occlusions are removed, a variant of Gappy Principal Component Analysis (Gappy PCA) is used to restore the full face from occlusion-free facial surfaces. Experimental results obtained on realistically occluded facial images from the Bosphorus 3D face database shows that, with the use of score-level fusion of regional Linear Discriminant Analysis (LDA) classifiers, the proposed method improves rank-1 identification accuracy significantly: from 76.12% to 94.23%.