Attention-based CNN and Relative Phase Feature Modeling for Improved Imagined Speech Recognition

In EEG (Electroencephalography) based imagined speech recognition research, deep learning is gradually being applied, but has not yet achieved sufficient performance. The main reasons for this are the lack of ability to acquire generalized feature representations and the insufficient amount of training data. For the former, we propose two methods, one is a CNN-based feature extractor that introduces an attention mechanism, and the other is an improved feature extractor that introduces the inter-channel relative phase obtained from the multi-channel EEG signal as an additional spectral feature instead of a CNN-based feature extractor. For the latter, we propose a pre-training method using multi-subject EEG data to construct subject-adapted models for an imagined speech recognition task. The effectiveness of the proposed methods is verified by using an imagined speech recognition task for English phonemes, classification for English vowels and consonants. As a result, the CNN model with attention mechanism successfully improved the feature extraction capability of the CNN and achieved a classification accuracy rate of 71.9%, which are approximately 20% higher than that of the CNN only. On the other hand, a simple neural network based classifier approach using spectral features as input achieved 71.1% by using the proposed inter-channel relative phase features over the conventional relative phase features, and 75.0% by applying ensemble processing to the test data. Pre-training and subject adaptation also achieved a 6% improvement in accuracy, indicating that it is possible to mitigate the impact of insufficient training data.