Robust realtime feature detection in raw 3D face images

3D face data contains holes, spikes and significant noise which must be removed before any further operations such as feature detection or face recognition can be performed. Removing these anomalies from the complete data is expensive as it also contains non-facial regions. We present a realtime algorithm that can detect the eyes and the nose tip in raw 3D face images in about 210 msecs. With three points, the data can be aligned to a canonical pose or registered to a reference face allowing the face area to be accurately cropped. The more expensive preprocessing steps can then be applied to the cropped region of the face only. We calculate the x and y gradients from the range image and train separate feature detectors in the three representations. Each detector is trained using the AdaBoost algorithm and Haar-like features. Haar features detect higher order discontinuities in the gradient images which form the core of the proposed algorithm. Multiple feature detections in the three images are clustered and anthropometric ratios are used to eliminate outliers. The centroids of the remaining candidates are used as feature points. Experimental results on the FRGC v2 database gave over 99% detection rates. Detailed quantitative analysis and comparison with the ground truth feature locations is provided.