Multi-Stream Deep Convolution Neural Network With Ensemble Learning for Facial Micro-Expression Recognition

Micro-expression recognition has gained much attention in research communities. Among its proposed solutions, deep learning approaches have shown promising results over the past few years. In this paper, we propose a multi-stream deep convolution neural network with ensemble classification for facial micro-expression recognition. The multi-stream network uses the deep features of a residual network, densely connected convolutional network, and visual geometry group. The features of these aforementioned architectures are extracted from their pooling layers and become very resource-intensive due to their high dimensions. The principal component analysis is applied to these features for their dimensionality reduction. Stacking, an ensemble classification technique, is performed on these deep features with three base learners (random tree, J48, random forest) and a meta learner (random forest). Experiments were performed using publicly available datasets, namely: CASME-II, CASME2, SMIC, and SAMM. The proposed approach (PA) is compared with twelve approaches. The results show that the PA outperformed the existing approaches in terms of accuracy and time efficiency.