Chinese Multimodal Emotion Recognition in Deep and Traditional Machine Leaming Approaches

in this paper, we propose our system in the MEC 2017 Chinese multimodal emotion recognition challenge which is based on the Chinese Natural Audio-Visual Emotion Database (CHEAVD). We extract various features in different modalities such as acoustic signals and facial expressions, and employ traditional machine learning and deep learning ways to recognize human emotions in the videos. For audio, we use two acoustic feature sets, IS09 and large emotion feature set, combined with traditional classifiers and DBN, respectively. For video, we take LBP-TOP feature as the input of SVM and also utilize CNN for extracting face expression features to train RNN to learn the temporal information. A decision-level average fusion method is applied to make prediction for both unimodal and multimodal emotion recognition. Experiment results on the challenge database show our effectiveness.