Deep Fusion of VGG19 and ConvMixer with Superlet Transform for Cognitive Load Detection

The cognitive load triggers neural activity, which is crucial for understanding the brain’s response to mental stress or stimuli that induce stress. Electroencephalogram (EEG) signals were collected from a mental arithmetic task (MAT), simultaneous task EEG workload datasets and segmented into 4-second intervals. These segmented signals were then transformed into images using various time-frequency conversion methods (TF), including the Short-Time Fourier Transform (STFT), Continuous Wavelet Transform (CWT), Q Transform (QT), and Superlet Transform (SLT). The resulting TF images were fed into convolutional neural networks (CNNs), such as VGG19, ConvMixer, and a fusion of VGG19+ConvMixer. CNN models were trained using the Adam optimizer to detect cognitive load. The preprocessing involved normalization, and scaling in both phases. Among the models tested, the SLT-based TF-EEG with the fusion of the VGG19 and ConvMixer model outperformed other TF conversion methods and CNN architectures. The VGG19+ConvMixer utilizes the individual advantages of both VGG19 and ConvMixer models. It helps reduce overfitting and vanishing gradients, enhances performance with new data, improves GPU acceleration, and reduces computational cost due to its simpler architecture. However, the SLT effectively handles non-stationary data through its adaptive multi-resolution approach, making it ideal for EEG analysis. The SLT + VGG19 + ConvMixer model achieved accuracies of 97.26% on the MAT dataset and 96.04% on the STEW dataset, with all other evaluation metrics such as precision, sensitivity, specificity, F-score, MCC, Jaccard index, and Cohen’s kappa exceeding 94%. These findings can enhance real-time cognitive load monitoring, benefiting areas such as personalized learning, mental health, and stress management by detecting cognitive load to improve performance and reduce stress in critical situations.