Cross-Subject Classification of Spoken Mandarin Vowels and Tones with EEG Signals: A Study of End-to-End CNN with Fine-Tuning | IEEE Conference Publication | IEEE Xplore

Cross-Subject Classification of Spoken Mandarin Vowels and Tones with EEG Signals: A Study of End-to-End CNN with Fine-Tuning


Abstract:

Direct speech brain-computer interface (DS-BCI) is an ideal way for speech communication by decoding signals collected from the brain. Electroencephalogram (EEG) has gain...Show More

Abstract:

Direct speech brain-computer interface (DS-BCI) is an ideal way for speech communication by decoding signals collected from the brain. Electroencephalogram (EEG) has gained widespread use in DS-BCI studies due to its simplicity of operation and high temporal resolution. However, as human brain exhibits considerable inter-individual variability, classification models trained on the basis of data from one subject may not generalise well to other individuals, which is a major challenge in existing EEG signal classification studies. In this paper, the cross-subject classification performance of spoken Mandarin speech with EEG signals was investigated by using an end-to-end convolutional neural network (CNN) model pretrained on the source data and fine-tuned on the target data. The raw EEG signals were directly used as the input to the model without using extracted features. In addition, adding Gaussian noise was used as the data augmentation method in order to deal with the unbalanced dataset. The proposed method was tested on a collected EEG dataset of spoken Mandarin speech, including vowel classification and tone classification tasks. The average classification accuracies of four vowels and four tones were 63.1% and 51.7% respectively. The average accuracy of tone classification was significantly improved compared with the machine learning and subject-dependent methods. The results of this work showed the potential of the fine-tuning based CNN model in the cross-subject studies of EEG decoding.
Date of Conference: 31 October 2023 - 03 November 2023
Date Added to IEEE Xplore: 20 November 2023
ISBN Information:

ISSN Information:

Conference Location: Taipei, Taiwan

Funding Agency:


References

References is not available for this document.