A Multi-Modality Attention Network for Driver Fatigue Detection Based on Frontal EEG, EDA and PPG Signals

Fatigue driving is a common issue that often leads to traffic accidents, which has motivated numerous automatic driving fatigue detection methods based on various sources, especially reliable physiological signals. However, it still faces the challenges of accuracy, robustness and practicality, especially for the cross-subject detection. The fusion of multi-modality data can improve the effective estimation of driving fatigue. In this work, we take the advantages of user-friendly and multi-modality signals to build a Multi-Modality Attention Network (MMA-Net) for driver fatigue detection with frontal electroencephalography (EEG), electrodermal activity (EDA) and photoplethysmography (PPG) signals for a hybrid. Specifically, a signal adaptive coding module (SAC-M) has been constructed to fully excavate spatial-temporal information of signals, combining with an attention-based feature dissimilation module (AFD-M) to further obtain key comprehensive features. In addition, the performances of baseline models and state-of-the-art methods on signal sources with different window lengths are also compared. The cross-subject experiment is performed on two groups of 14 participants in the driving simulation experiment. The experimental results prove the superiority of our proposed method. It is possible to use the MMA-Net for driver fatigue detection with user-friendly multi-modality signals such as our selected frontal EEG, EDA and PPG in real-world applications.