Multi-Representation Convolutional Neural Network to Recognize Human Emotion

Emotion recognition using brain-computer interfaces (BCIs) is an emerging field with the ongoing challenge of developing robust and efficient classification methods. This study introduces an original approach, the multi-representation convolutional neural network (MrCNN), designed to exploit the complementary features of recurrence plots (RP) and spectrograms. The efficacy of this new method was empirically evaluated and compared with the standalone RP and spectrogram methods in emotion classification tasks using electroencephalogram (EEG) signals. Two widely used datasets, DEAP and SEED, were used as the experimental platforms for this study. Employing a multifaceted analytical approach involving descriptive statistics, data visualization, and hypothesis tests, this study revealed that MrCNN significantly outperforms the standalone RP and spectrogram methods in terms of key performance metrics. Specifically, MrCNN achieved an accuracy of 0.9663, sensitivity of 0.9848, specificity of 0.9825, kappa score of 0.9511, and F1 score of 0.9656 for the DEAP dataset. For the SEED dataset, the accuracy was 0.9695, with sensitivity, specificity, kappa scores, and F1 score of 0.9844, 0.9842, 0.9543, and 0.9695, respectively. These results were statistically validated, and a comparative analysis with existing literature substantiated the robustness and superiority of the MrCNN method. This study sets new performance benchmarks and offers a reliable and effective approach for emotion classification using EEG signals.