A joint facial point detection method of deep convolutional network and shape regression

Facial landmark detection is a challenging task with broad applications. Many approaches have been proposed with varying degrees of success. Regression based methods update the facial point positions iteratively. The mean shape or shapes sampled from training set is often used as the initialization, which sometimes may lead to a local minimum in update due to the offset of initial positions and target positions. On the other hand, convolution network based method shows superior accuracy, at the cost of a complex and unwieldy architecture of deep model. To address the above limitations, a new approach for facial landmark detection is proposed in this paper. The key idea is to combine deep convolution networks with shape regression approach. Deep convolution networks in the first level provide a highly robust initial shape, while the following regression finely tunes the initial prediction to achieve high accuracy. Extensive experiments show that the regression based methods are very sensitive to initializations and the proposed approach (i) achieves high accuracy on public datasets, especially outperforms existing methods in challenging conditions like large pose expression variation, and (ii) reduces model complexity drastically compared to previous methods.