Spatial-Temporal Constraint Learning for Cross-Subject EEG-Based Emotion Recognition

Recent researches combine domain adaptation methods with elaborate feature extractors to better learn domain-invariant and discriminative features for cross-subject Electroencephalogram(EEG)-based emotion recognition. Existing models only utilize domain adaptation to constrain spatial learning or temporary learning, though the domain shift will possibly appear in both spatial and temporal learning stages. And some models simply treat the different subjects in the source domain as a whole, ignoring the data structure of the source domain. Motivated by the above problems, we design a novel model, Spatial-Temporal Constraint Learning (STCL), which adopts the Multi-Layer Perceptron (MLP) and Transformer Encoder for spatial and temporal features learning, respectively. In the spatial learning stage, we design Multi-Subject Prototypes Alignment (MSPA), which treats different subjects as different domains. In the temporary learning stage, we utilize the adversarial training strategy which treats different subjects in the source domain as a whole to further narrow down the domain gap. In addition, to improve the representative ability of our model in the target domain, we put forward Target Samples Selective Strategy (TSSS) which selects the samples from the target domain with reliable pseudo-labels for training STCL. The cross-subject experiments on two benchmark datasets have demonstrated the effectiveness of our model. Our model achieves the accuracies of 83.43 %, 78.36 %, and 80.09 % for three sessions respectively on SEED, 60.51 % for valence classification, and 63.68 % for arousal classification on DEAP.