Dual AxAtGAN: A Feature Intregrate BCI Model for Image Reconstruction

The visual stimulus-oriented brain activity reconstruction method based on BCI technology has the advantages of high transmission rate and easy operation. However, the problem of losing feature information due to vector dimensionality reduction still exists in existing visual evoked EEG reconstruction algorithms. This motivates us to design an end-to-end model, Dual AxAtGAN, where the network is nested using two Dual GAN as the basic framework, which is able to directly perform the task of reconstructing from video-evoked EEG to the corresponding video. In addition, existing reconstruction algorithms are based on convolutional neural networks, which can only focus on a local subset of a pixel point in an image, and leads to the model's lack of understanding of the long-range dependence of the EEG signal features, and consequently it is unable to reconstruct the corresponding image or video. To address the above challenges, inspired by the transformer block, we modified the axial multi-head self-attention module instead of the CNN blocks, which improves the model's understanding of the long-range dependence of EEG signals. In addition, we train the data from two branches, global and local, which in turn improves the performance of the model, and finally reconstruct the videos corresponding to the evoked EEG. To test the reconstruction effect of the model, we design a supervised paradigm that meets the requirements of this experiment to obtain the dataset of video-evoked EEG. Compared with the existing methods, our proposed model can better perform the task of video-evoked EEG reconstruction.