Deep tree-structured face: A unified representation for multi-task facial biometrics

Automatic facial image analysis has received considerable research interests due to its important role in computer vision and biometrics. As the key component, face feature is usually extracted under largely controlled environment and learnt for specific tasks which limits its discriminant capability in a multi-task learning scenario. In this paper, we present a novel deeply learnt tree-structured face representation to model the human face with multiple semantic meanings, such as identity, expression and age, that wouldyield a unified feature representation of the facial image. The tree structure is built based on the incorporation of an unsupervised shallow network that generates the low-level features serving as the leaf nodes and the recursive application of the designed semi-supervised AutoEncoder to generate the intermediate and root nodes. By incorporating the label information with different semantic meanings, the designed semi-supervised AutoEncoder aims to distinguish the latent factors embedded in facial images with automatically learned tree structure and weights. To validate the effectiveness of the proposed facial representation, we design comprehensive experiments based on the FACES dataset which is considered as the most challenging benchmark that reflects multiple biometric factors. We show that the proposed feature yields unified representation in multitask facial biometrics. The multi-task learning framework is applicable to many other computer vision tasks.