A Combinational Deep Learning Approach to Visually Evoked EEG‐Based Image Classification

Summary

Visual stimulus evoked potentials are neural oscillations acquired, from the brain's electrical activity, evoked while seeing an image or video as stimuli. With the advancement of deep learning techniques, decoding visual stimuli evoked EEG (electroencephalogram) signals has become a versatile study of neuroscience and computer vision alike. Deep learning techniques have capability to learn problem specific features automatically, which eliminates the traditional feature extraction procedure. In this work, the combinational deep learning–based classification model is used to classify visual stimuli evoked EEG signals while viewing two class images (i.e., animal and object class) from MindBig and Perceive dataset without the need of an additional feature extraction step. To achieve this objective, two deep learning–based architecture have been proposed and their classification accuracy has been compared. The first proposed model is a modified convolutional neural network (CNN) model and the other one is hybrid CNN+LSTM model which can better classify and predict the EEG features for visual classification. The raw EEG signal is converted to spectrogram images as CNN‐based networks learn more discriminating features from images. The proposed CNN+LSTM‐based architecture uses a depthwise separable convolution, i.e., Xception Network (Extreme inception network) with a parallel LSTM layer to extract temporal, frequential and spatial features from EEG signals, and classify more precisely than proposed CNN network. The overall average accuracy achieved are 69.84% and 71.45% on CNN model and 74.57% and 76.05% on combinational CNN+LSTM model on MindBig and Perceive dataset, respectively, which shows better performance than CNN model.