Enhanced Online Continuous Brain-Control by Deep Learning-Based EEG Decoding

Objective: A growing amount of deep learning models for motor imagery (MI) decoding from electroencephalogram (EEG) have demonstrated their superiority over traditional machine learning approaches in offline dataset analysis. However, current online MI-based brain-computer interfaces (BCIs) still predominantly adopt machine learning decoders while falling short of high BCI performance. Yet, the generalization and advantages of deep learning-based EEG decoding in realistic BCI systems remain far unclear. Methods: We conduct a randomized and cross-session online MI-BCI study on 2D center-out tasks in 15 BCI-naive subjects. A newly proposed deep learning model named interactive frequency convolutional neural network (IFNet) is leveraged and rigorously compared with the prevailing benchmark namely filter-bank common spatial pattern (FBCSP) for online MI decoding. Results: Through extensive online analysis, the deep learning decoder consistently outperforms the classical counterpart across various performance metrics. In particular, IFNet significantly improves the average online task accuracy by 20% and 27% in two sessions compared with FBCSP, respectively. Moreover, a significant cross-session training effect is observed by the IFNet model ( ${P}={0}.{017}$ ) while not for the controlled method ( ${P}={0}.{337}$ ). Further offline evaluations also demonstrate the superior performance of IFNet over state-of-the-art deep learning models. Moreover, we present unique behavioral and neurophysiological insights underlying online brain-machine interaction. Conclusion: We present one of the first studies about online MI-BCIs using deep learning, achieving substantially enhanced online performance for continuous BCI control. Significance: This study suggests the good utility of deep learning in MI-BCIs and has implications for clinical applications such as stroke rehabilitation.