Integration of Transfer Learning and Self-Attention for Spontaneous Micro-Expression Recognition

Micro-expressions are involuntary facial expressions which cannot hide true mental state of a person. These expressions appear for a very short duration of time and thus are very difficult to spot and recognize. Performance of deep learning models becomes stagnant due to overfitting; transfer learning helps to address this issue. Although, spotting and classification of micro-expression is a challenging task, but self-attention mechanism can easily detect relevant facial region required for micro-expression recognition. Considering these advantages of transfer learning and self-attention mechanism, in this work, two deep CNN models - Dual Stream ResNet18 (DSResNet) and Dual Stream ResNet18 Attention (DSResNetAtt) are proposed which exploits transfer learning through ResNet18 and self-attention for spontaneous Micro-expression recognition (MER). To demonstrate the effect of self-attention, the model is evaluated in terms of classification accuracy on three benchmark datasets CASME-I, CASME-II, and SAMM. Performance of the model has also been compared with state-of-the-art existing models and comparative analysis shows its efficiency in terms of classification accuracy.