Micro-expression Recognition Based on Deep Mutual Learning Network

Facial micro-expressions, indicating brief and subtle facial movements, are challenging to analyze. How to fully explore the discriminant and subtle features with only a limited number of samples for deep learning training has not been well studied. To leverage knowledge acquisition in deep learning models, this paper adopts the strategy of deep mutual learning to guide the interactive training of micro-expression sequences between different modes. This method uses the lightweight network Resnet-18 which is suitable for small-scale database as the backbone network. Then, the dual-stream structure is introduced to explore the multi-mode representation for ME recognition. Specifically, the main network is based on optical flow and the auxiliary network is based on RGB sequence. In this method, the difference image based on key frame is used to obtain the relevant feature representation of the micro-expression input by the module. In the training stage, the training process is optimized by mutual learning loss (including supervised learning loss and interaction loss), so that the prediction of each model is consistent with the real information of the training sample and the prediction of the other model. In the testing phase, we use optical flow to test separately to enhance the recognition ability of optical flow branches. Therefore, we can prune RGB branches on the premise of ensuring the accuracy to improve the recognition speed. The experimental results on public micro-expression data sets show that the proposed method can effectively improve the recognition accuracy and the overall performance is better than the state-of-the-art methods.