Expression Recognition Based on Multi-Scale Adaptive Parallel Integration Network

Facial expressions are the most direct outward expression of an individual’s emotion. Unlike other pattern recognition problems, the feature differences between facial expressions are slight. The richness of feature extracted by a single convolutional neural network is poor, and it is difficult to describe the feature differences effectively. Therefore, this paper proposes an expression recognition based on a multiscale adaptive parallel integration network. First, two depthwise separable convolutions of different sizes are used to design sub-networks 1 and 2, and a sub-network3 with a self-attention mechanism is introduced. The small, large convolutional kernel and the self-attention mechanism can extract local detail features, edge contour features, and global features of facial expressions, respectively. Then, this paper designs a sub-network adaptive weight generation mechanism that evaluates each sub-network’s accuracy and dynamically assigns weights to it. Thus, the parallel integrated outputs of sub-networks 1, 2, and 3 are effectively realized, and the differences between facial expressions can be better described. Finally, the proposed network is tested on FER2013, RAF-DB datasets. Experimental results show that our proposed method is comparable to the most advanced methods with 88.17% on RAF-DB, and 74.94% on FER2013.