Multiscale Global Prompt Transformer for EEG-Based Driver Fatigue Recognition

Driver fatigue is a critical factor that lead to traffic accidents with a high fatality rate. Electroencephalogram (EEG) is one of the most reliable indicators to objectively assess fatigue status, but recognizing fatigue driving status from it is still an essential and challenging problem. In this paper, we propose a multiscale global prompt Transformer (MsGPT) deep learning model, which can automatically recognize driver fatigue end-to-end. First, we construct an intra-inter-scale cascade framework based on Transformer with a multiscale convolutional patch embedding (MC-PatchEmbed), and guide global-local feature interaction by adding a global prompt token throughout. Second, to efficiently integrate intra-scale and inter-scale feature information, we design a mixed token by aggregating the output from the intra-scale, which includes rich low-level feature information for multiscale. Moreover, a novel learnable query is introduced into multi-head self-attention (MSA) to reduce the computational complexity to linear level. Experiments are conducted on the SEED-VIG dataset and the SADT dataset with both intra-subject and inter-subject settings to evaluate the performance of MsGPT, and the results show that MsGPT greatly outperforms various methods in terms of the classification evaluation metrics of EEG-based fatigue driving.
Note to Practitioners
—This paper considers the use of raw EEG data to recognize the driver fatigue state. Existing methods mainly rely on manually extracted EEG features and convolutional neural network (CNN) based inference. However, the large intra-individual and inter-individual differences greatly limit the extraction of EEG fatigue features. This paper suggests a multiscale global prompt Transformer (MsGPT) deep learning model. This model leverages a shared weighting mechanism to construct an inter-to intra-scale multiscale framework that can capture refined fatigue features not achievable at a single scale, we incorporate a new Transformer of the global prompt mechanism, which facilitates multiscale local-to-global fusion of long-term physiological signals. Our experiments demonstrate the superiority of our method on two datasets with intra-subject and inter-subject settings. The proposed method can be readily deployed in the automatic driving assistance system to alert drivers to avoid or reduce traffic accidents caused by excessive fatigue.