Representation Learning Based on Variational Autoencoders for Imagined Speech Classification

Brain computer interfaces based on speech imagery have attracted attention in recent years as more flexible tools of machine control and communication. Classifiers of imagined speech are often trained for each individual due to individual differences in brain activity. However, the amount of brain activity data that can be measured from a single person is often limited, making it difficult to train a model with high classification accuracy. In this study, to improve the performance of the classifiers for each individual, we trained variational autoencoders (VAEs) using magnetoencephalographic (MEG) data from seven participants during speech imagery. The trained encoders of VAEs were transferred to EEGNet, which classified speech imagery MEG data from another participant. We also trained conditional VAEs to augment the training data for the classifiers. The results showed that the transfer learning improved the performance of the classifiers for some participants. Data augmentation also improved the performance of the classifiers for most participants. These results indicate that the use of VAE feature representations learned using MEG data from multiple individuals can improve the classification accuracy of imagined speech from a new individual even when a limited amount of MEG data is available from the new individual.