Residual Attention Capsule Network for Multimodal EEG- and EOG-Based Driver Vigilance Estimation

Driver vigilance estimation is essential for fatigue and traffic accident reduction. Although previous algorithms for driver vigilance estimation have achieved high evaluation performance, several challenges remain open: 1) how to effectively utilize multimodal electroencephalography (EEG) and electrooculogram (EOG) signals remains challenging; 2) how to capture vector feature information in part-to-whole space more effectively; and 3) how to enhance the core ability of a network, including in-depth mining of features and eliminating redundant information as much as possible. To address these challenges, we propose a novel multimodal detection method for driver vigilance estimation, which consists of residual attention blocks and a capsule attention mechanism. Specifically, we propose a residual attention network to optimize the channel-space features of low-level capsules to adaptively rescale the features of each channel to fuse the interdependence among feature channels. Meanwhile, the part–whole relationship in the features is explored with the use of dynamic routing, the features with large contributions are strengthened, and the transmission efficiency of the neural network is ultimately improved. The experimental datasets show that the residual attention capsule network (Res-att-capsnet) outperforms the state-of-the-art baselines in both regression and classification tasks. The feasibility and effectiveness of our proposed method are thus demonstrated.