Multi-Class Image Generation from EEG Features with Conditional Generative Adversarial Networks

Exploring the visual coding mechanism of the human brain has been an important topic in brain-computer interface technology, holding the potential to provide new insights into the research of the Internet of Things (IoT). Especially, the use of brain signals for image generation has attracted people’s attention. In this paper, we propose a multi-class image generation method based on electroencephalography (EEG) features with conditional generative adversarial networks (CGAN). Specifically, we synchronously collect the EEG signals when subjects watch visual stimuli through EEG equipment and construct a multi-category image-EEG dataset. The obtained EEG dataset undergoes preprocessing to eliminate interference from electrooculographic signals and electromyographic signals. Based on this dataset, we extract EEG features related to visual stimuli by deep learning algorithms and integrate them with CGAN for image generation. We employ different EEG feature extraction methods for image generation and conduct comparative experiments. The experimental results demonstrate that the proposed method based on the CGAN model utilizing the joint EEG spatio-temporal and temporal-frequency features can achieve better performance in the image generation task.