A Study of Deep Learning Based Classification of Mandarin Vowels Using Spoken Speech EEG Signals

Brain computer interface (BCI) based on imagined speech provides an alternative access for human-computer interaction and can help patients who are unable to speak to communicate with the outside world. However, the complexity and variability of speech-related electroencephalographic (EEG) signal make decoding speech information from EEG signals a challenging task. This study aimed to explore feature extraction methods and classifiers that were suitable for speech-related EEG classification. Six feature extraction techniques and three classifiers were used to classify spoken speech EEG signals of four Mandarin vowels. The classification results showed that the accuracy reached 68.7% when using channel cross-covariance (CCV) with Riemannian manifolds as input features and deep convolutional neural network (Deep ConvNet) as the classifier, which was better than classification performance of other methods and previous studies. This study provides insights into the effectiveness of various feature extraction and classification methods in EEG-based speech classification. It suggests the potential of CCV combined with Riemannian manifold features and a Deep ConvNet based classifier for speech imagery studies.