Structure Representation With Adaptive and Compact Facial Graph for Micro-Expression Recognition

The subtle and slight motions of micro-expressions (MEs) leave few effective features to micro-expression recognition (MER), making MER a challenging task. Existing works mainly focus on constructing strong representations from entire videos, individual frames, or redundant structural graphs, however, spatial structure feature learning of MEs leaves much space for further improvement. To solve the issue, this paper introduces a novel two-stream network for MER without any prior knowledge called Focusing on Few Discriminative Information Network (FFDIN). Specifically, in the temporal stream, the difference between the Apex and Onset frames is utilized as input to reduce redundant information and aggregate temporal information. Meanwhile, spatial attention is incorporated into the CNN stream to encourage the network to focus on salient features. In the structural stream, the Adaptively Select Strategy (ADSS) is proposed to automatically locate few effective regions of MEs by selecting the strong long-term dependent cropped patches and corresponding adjacency matrix. Then, the Graph Nodes Generation (GNG) module is designed to capture local and global information in tiny cropped patches and project the feature maps into graph nodes. Extensive experiments conducted on the CASME 2, SAMM, and SMIC datasets demonstrate that the proposed network can achieve superior performance than the state-of-the-art methods.