Research on Emotion Recognition Based on Multi-Modal Physiological Signals

The research on emotion recognition based on multi-modal data fusion, including walking gait, expression/micro-expression, voice and eye movement features, is one of the current research hotspots. However, since the above behavioral features are camouflageable, the emotion recognition results are often difficult to represent the most authentic emotional state. Physiological signals, such as Electrocardiogram (ECG), Electroencephalogram (EEG) and Galvanic Skin Reaction (GSR), are spontaneous electrophysiological responses of the human body. They are not camouflaged and can objectively reflect the real emotional state. Therefore, emotion recognition based on physiological signals is of great application value. In this paper, we respectively extract features from ECG, EEG and GSR signals, and build the depression detection model based on traditional machine learning methods and the emotion recognition model based on graph convolution neural network. The experimental results show that the recognition accuracy of the depression detection model on the depression database can reach 93.51%, and the recognition accuracy of the emotion recognition model based on graph convolution neural network on the SEED emotion recognition database can reach 85.30%, which is superior to the existing emotion recognition algorithms. This paper makes exploring research on emotion recognition and depression detection. The multi-modal physiological fusing scheme and the proposed machine learning modal is proved to be effective for wearable device in home environment.