A Siam-LSTM Model for Multi-Channel EEG on VR Motion Sickness Recognition

The combination of prior knowledge of Electroencephalography (EEG) and deep learning always receives better results in EEG pattern recognition. Thus, we propose end-to-end models combining Functional Brain Network (FBN) and Siamese Long Short-Term Memory model (Siam-LSTM) and apply them to a Virtual Reality Motion Sickness (VRMS) recognition task. Siam-LSTM is a key module in the proposed models to process EEG signal electrode by electrode via the shared layers. The models apply 1D-CNN and Siam-LSTM to extract temporal features and then Pearson correlation coefficients of the features among the electrodes are computed as the FBNs. The 2D-CNN and dense networks are used to extract the FBN feature and do regression analysis. In final a VRMS-related EEG dataset is used to verify the models by 10-fold cross-validation, while the VRMS level is assessed by the subjective simulator sickness questionnaire (SSQ). The experimental results show that the two major proposed models named BC-CNN-LSTM and BC-CNN-LSTM2 obtain the best performance in regression, i.e., mean square errors between the predicted SSQ scores and the real SSQ scores are 26.89±8.89 and 24.75±4.70 respectively. The proposed models provide a new idea for LSTM to process multi-electrode EEG signals and outperform the state-of-the-art methods in VRMS recognition.