A 4D LSTM network for emotion recognition from the cross-correlation of the power spectral density of EEG signals

Emotion measurement with electroencephalography (EEG) and their recognition through analytical models is based on the theory that emotions have specific patterns in cortical activation. The objective of our work is to build a Deep Learning (DL) architecture to discover these activation patterns from the Power Spectral Density (PSD) of the EEG signals from the public data set from the Swartz Center for Computational Neuroscience (University of California San Diego) of self-induced emotions.In detail, for each subject, we divided the EEG temporal signals into partially overlapping "epochs" (segments of EEG signals with the same length based on experimental parading) and for each epoch we computed the PSD. The spectrogram was divided into the six spectrograms of the frequency bands of interest defined in the literature (delta, theta, alpha, sigma, beta, gamma). For each frequency band, the cross-correlation of each epoch with the following epochs was computed and related to their spatial position in the scalp.A 4D LSTM network, based on 2D convolutional layers, is used to recognize the emotional states, using spatial information (EEG channel positions), also by exploiting the temporal information (evolution over time) of the correlation among epochs. Preliminary results have been obtained by recognizing the "disgust" and the "relax", both intra- and inter-subject. The results show the convergence of the network but the accuracy of prediction on subjects varies greatly from subject to subject, in agreement with the most recent literature. Moreover, though intra-subject accuracy was sufficiently high for some subjects, the inter-subject accuracy was very low, thus also demonstrating the subjective nature of the emotional patterns. Though deeper analysis is required, the paper presents a new model for the processing of EEG emotional signals.