Frontalization with Adaptive Exponentially-Weighted Average Ensemble Rule for Deep Learning Based Facial Expression Recognition

Automatic Facial Expression Recognition (FER) is an important technique in human-computer interfaces and surveillance systems. It classifies the input facial images into one of the basic expressions (e.g., anger, sad, surprise, happy, disgust, fear, and neutral) and has attracted significant attention in pattern recognition and computer vision. In this paper, we proposed an advanced convolutional neural networks based FER system. It applies the techniques of face frontalization and feature positioning to reduce the effects of background noise and non-prominent parts. Moreover, the hierarchical structure together with the adaptive exponentially weighted average ensemble are adopted to further improve the accuracies. Simulations on several datasets show that the proposed system outperform state-of-the-art FER methods and can well identify the expression of a person.