Analysis of Motor Imagery EEG Signals Based on Multi-Modal Spatio-Temporal Contrastive Learning

The progress of electroencephalogram (EEG) motor imagery classification has received widespread attention from the fields of human-machine interactions and cognitive science in recent years. Moreover, existing classification methods lack consideration for multi-modal data features. To address the issue, this paper proposes a multi-modal spatio-temporal feature contrastive learning model (MSTC). Initially, raw EEG signals are transformed into a multi-modal representation, with the brain map modal generated by STFT and the signal modal through data transformation. Additionally, our multi-modal feature extraction model is utilized to enhance the extraction capabilities for multi-modal features. Furthermore, it utilizes contrastive learning to maximize the similarity of features in positive pairs with the same labels. Finally, this model imports the parameters learned from pre-training into the fine-tuning stage for motor imagery classification. Experiments have been conducted on two public EEG datasets. To evaluate the effectiveness of our architecture, both ablation studies and control experiments have been executed. The results demonstrate that training an MI classifier using features learned from our proposed MSTC achieves performance comparable to supervised training, thus broadening the potential for practical applications in this field.