Displacement Generating Module Based End-to-end Micro-expression Recognition Network

With the rapid development of deep learning, the research and application of micro-expression recognition are more and more extensive. In the existing solutions, the methods using the difference between onset frame and apex frame have high accuracy and low computational cost. But their extraction of dynamic features is not integrated with classification network, resulting in the lack of feedback from classification loss to the extracted dynamic features. In this paper, we propose a novel Displacement Generating Module (DGM) which uses convolution module to generate the displacement feature between onset frame and apex frame instead of traditional optical flow or dynamic image. The new DGM is integrated with the existing LEARNet to form an end-to-end micro-expression recognition network, where the classification loss can feed backward to the parameters of DGM to obtain better displacement features. We also present a random selection method of apex frame to increase the amount of training data, and present a normalization operation for the displacement features with different scales. Our new approach has been tested on SAMM, SMIC, CASME II datasets with LOSO evaluation method, and achieves 0.737 on UF1 and 0.726 on UAR, which is obviously higher than existing networks with optical flow and dynamic imaging techniques.