Privileged Information-Based Conditional Structured Output Regression Forest for Facial Point Detection

This paper introduces a regression method called Privileged Information (PI)-based Conditional Structured Output Regression Forest (RF) for facial point detection. To train RF more efficiently, the method utilizes both PI, that is side information that is available only during training, such as head pose or gender, and shape constraints on the location of the facial points. We propose selection of the test functions at some randomly chosen internal tree nodes according to the information gain calculated on the PI. In this way, the training patches that arrive at leaves tend to have low variance both in terms of their displacements in relation to the facial points and in terms of the PI. At each leaf node, we learn three models: first, a probabilistic model of the pdf of the PI; second, a probabilistic regression model for the locations of the facial points; and third, shape models that model the interdependencies of the locations of neighboring facial points in a predefined structure graph. The latter two are conditioned on the PI. During testing, the marginal probability of the PI is estimated and the facial point locations are estimated using the appropriate conditional regression and shape models. The proposed method is validated and compared with very recent methods, especially that use Regression Forests, on datasets recorded in controlled and uncontrolled environments, namely, the BioID, the Labeled Faces in the Wild, the Labeled Face Parts in the Wild, and the Annotated Facial Landmarks in the Wild.