Assessment of Driver Inattention State using Multimodal Attentive Feature Fusion

Driver inattention is a leading cause of traffic accidents. Multimodal physiological based driver inattentions plays an important role. In recent years, textile electrodes based wearable have been extensively explored in detecting the driver inattention states. However extraction of significant biomarkers and features for accurately detecting driver inattention is still challenging. In this study, we evaluated combination of self and cross attention mechanisms for assessing driver inattention using textile wearable multimodal data. Electrocardiography (ECG) and Respiration (RSP) data were collected from 20 healthy volunteers wearing smart shirts in two driving scenarios. We extracted time, frequency, nonlinear, and statistical features for ECG, and frequency and statistical features for RSP. These features are fed to 1D convolutional layers (CL) networks. The attention mechanisms were used to fuse attentive CL features to classify inattention states. Experiments are performed on varied attention mechanism. The proposed approach is able to classify driver inattention states. The combinations of cross attention followed self attention mechanism based features, achieving an average Accuracy of 85% and an average F-measure of 80% respectively. The second best results found to be using cross attention mechanism. The integration of the attention mechanism facilitated the capture of attentive feature information in real time scenario.