A Hybrid End-to-End Spatiotemporal Attention Neural Network With Graph-Smooth Signals for EEG Emotion Recognition

Recently, physiological data such as electroencephalography (EEG) signals have attracted significant attention in affective computing. In this context, the main goal is to design an automated model that can assess emotional states. Lately, deep neural networks have shown promising performance in emotion recognition tasks. However, designing a deep architecture that can extract practical information from raw data is still a challenge. Here, we introduce a deep neural network that acquires interpretable physiological representations by a hybrid structure of spatiotemporal encoding and recurrent attention network blocks. Furthermore, a preprocessing step is applied to the raw data using graph signal processing tools to perform graph smoothing in the spatial domain. We demonstrate that our proposed architecture exceeds state-of-the-art results for emotion classification on the publicly available DEAP data set. To explore the generality of the learned model, we also evaluate the performance of our architecture toward transfer learning (TL) by transferring the model parameters from a specific source to other target domains. Using DEAP as the source data set, we demonstrate the effectiveness of our model in performing cross-modality TL and improving emotion classification accuracy on DREAMER and the emotional English word (EEWD) data sets, which involve EEG-based emotion classification tasks with different stimuli.