Fatigue and Distracted Driving Recognition Method Based on Multimodal Information Fusion

Fatigue and distracted driving are two of the leading causes of major accidents. Drivers play a crucial role in automobile safety, and accurately detecting their driving states can significantly enhance the safety of urban road traffic and improve road operation efficiency. Currently, mainstream research relies on visual features, EEG (Electroencephalography), EOG(Electrooculography), and EMG (Electromyography) to identify driving status. However, these methods are not easily scalable due to the high cost of the data acquisition devices and the fact that physical contact with the body can interfere with normal driving functions. This study conducted a simulation experiment using a driving simulator and physiological wristbands, with a highway as the experimental scenario. Data on the physiological and psychological parameters, driving performance, and wrist movement information of 33 drivers were collected. Firstly, features were extracted through time, frequency domain, and nonlinear analyses, and non-parametric tests were performed to analyze the differences in these features under different driving states, thus selecting an effective feature subset. Secondly, based on this, an impaired driving state recognition model was established by combining feature recursive elimination and machine learning, and the impact of different modal inputs on the model performance was analyzed. Finally, SHAP (SHapley Additive exPlanations) interpretable machine learning was employed to analyze the model results in depth. The results indicate that the XGBoost model based on multimodal information input performs the best, with accuracy, precision, F1 score, and recall reaching 94.28%, 94.28%, 94%, and 94.27%, respectively, demonstrating its effectiveness in recognizing driving status. The mean X-axis angular velocity of wrist motion is a key feature for identifying a driver’s driving status. Additionally, the mean X-axis angular velocity, mean lane offset, and mean X-axis acceleration are positively correlated with the probability of fatigued driving, while the standard deviation of yaw rate is positively correlated with the likelihood of a distracted driving state. This study is the first to use a fusion of physiological data, driving performance, and wrist movement information to identify impaired driving states. This method takes into account various impaired driving states and provides a non-invasive, robust, and real-time approach, which has practical significance.