3D Facial Landmark Localization Based on Two-Step Keypoint Detection

We propose an approach to 3D facial landmark localization based on two-step keypoint detection. In this paper, the facial landmarking problem is devided into two subproblems that are solved independently, keypoint detection and labeling. The two steps of the keypoint detection are as follows. First, coarse position of landmarks is located by using Supervised Descent Method on depth images. Extracting the neighbourhood of landmarks' coarse position as the valid keypoint region. Second, keypoints are detected by screening the subset of facial points within the valid keypoint region combining multiple local descriptors. In the stage of labeling, combinations of landmark candidates are generated from keypoints and the one fits the facial landmark model best is labeled as the landmark combination. On the FRGC v2.0 database, we achieve mean errors from 2.85 mm to 3.81 mm for each landmark. The overall detection success rate is 96.8%, among which 97.6% for neutral expression, 97.2% for mild, 93.3% for extreme. On the Bosphorus database, we respectively achieve the success rate of 92%, 95% and 88% under three different poses. Experimental results show that the presented approach is comparable to state-of-the-art methods in terms of its accuracy and good robustness is achieved against expression and small pose variation.