Facial Expression Recognition for Different Pose Faces Based on Special Landmark Detection

Facial expression recognition is a challenging task in computer vision field using only single facial image. As we know, human faces are convex spheres. The self-occlusion phenomenon generated from face pose will seriously affect the accuracy of expression recognition. In order to solve this problem, we propose a novel facial expression recognition method for different pose faces based on special landmark detection (FER-MPI-SFL). Our method is based on two shared networks. The outputs of the first Network are 29 special landmarks and 1 face box, which are the inputs of the second network and used to estimate face pose. The methods of RoIAlign and feature map concatenation are introduced in the second network to recognize the facial expression. The weight allocation of feature maps concatenation is guided by the result of pose estimation. In addition, an improved center loss is proposed to make the distances between the features of different expressions larger and easier to be classified in the feature space. As a result, superior performance to other state-of-the-art methods is achieved in facial expression databases CK+, MMI, Oulu-CASIA VIS and a new created database CASIA-MFE which contains more faces with different poses.