Pose-Independent Facial Action Units Recognition with Attention Enhanced Residual Mapping

Facial action units (AU) recognition is an essential issue of affective computing, which is important to modern human-computer interaction and virtual reality. Recent advances in deep learning have shown great achievements in facial action unit recognition. However, the conventional approaches are sensitive to the pose of head. To tackle this limitation, we propose a pose-independent AU recognition approach based on attention enhanced deep residual mapping. In the deep feature space, the non-frontal face is mapped to frontal face through attention enhanced residual addition to improve the performance of non-frontal AU recognition. The network consist of three parts: the base network, the residual mapping module and the channel attention enhanced module. The base network is the fine-tuned VGG-Face which are trained with frontal faces. Then, the residual mapping and channel-wise attention mechanism are proposed and introduced into the deep feature space to learn the AU consistent features of faces in different poses. The non-frontal facial features are combined with the residual to map it to frontal face. The channel-wise attention mechanism enables the network to understand which features are more important for the facial pose mapping process. We have demonstrated the effectiveness of our approach on FERA2017 dataset. The experiment results have shown that our approach has improved the face recognition performance.