Combining Attention Mechanism and Dual-Stream 3D Convolutional Neural Network for Micro-expression Recognition

Society values micro-expressions, which are a type of facial expression that shows the true feelings that people try to hide, because they might provide potentially important information in crime detection, lie detection, and other areas. Micro-expressions differ from conventional facial expressions in that they are involuntary, have a short duration, and have a small amplitude. As a result, it is difficult to distinguish micro-expressions artificially, the correctness is impossible to guarantee, and it is dependent on professional knowledge. This research proposes a micro-expression recognition method that combines an attention mechanism with a dual-stream 3D convolutional neural network to solve this challenge. The optical flow method is used to model the action information of micro-expression videos to highlight the micro-expression spatial motion features. First, the micro-expression dataset is expanded by adding 3D BN layer batch normalization to perform data normalization, and then the optical flow method is used to model the action information of micro-expression videos to highlight the micro-expression spatial motion features. Second, to compensate for the loss of facial information caused by the optical flow method, the face feature map and optical flow feature map are passed into the network model as input for deep spatio-temporal feature extraction. Third, the attention module combines 3D temporal and 3D spatial attention to help the model pay greater attention to the time periods and places where micro-expressions occur during feature extraction. In this paper, simulation experiments were conducted on the open dataset CASME II, and an accuracy of 68.57% was achieved, which is better than the existing methods.