Deep Learning-Based Real-Time Driver Cognitive Distraction Detection

Driver distraction is one of the main causes of traffic accidents. While there are different types of distraction (manual, visual, cognitive), cognitive distraction is particularly challenging, being only partially related to visual features detectable through cameras or an eye tracker system. Moreover, since cognitive distraction is not a point in time phenomenon, spotting this kind of distraction requires the processing of a certain time interval, which poses a further challenge for real-time performance. After a data collection campaign with N =42 subjects undertaking a twenty-question task (TQT) in a driving simulator, we developed a driver cognitive distraction detection system, with the goal of filling in some key gaps we identified in the literature towards real world deployment. First, we assessed the effectiveness of state-of-the-art time series-oriented deep learning models in learning features from 60 Hz raw input signals, thus implementing an end-to-end machine learning approach, without manual feature engineering. We demonstrated that such models are able to classify time-windows as small as 0.5 seconds, and are also more robust to sensor failures. Second, also with the support of AI explainability, we showed that processing vehicular data is fundamental to ensure performance, while physiological signals provide a less important, but still useful, contribution. Third, through a between- and within-subject design comparison, we showed that eye-tracker and, particularly, physiological signals are much more prone to inter-individual variability, thus overfitting. This is fundamental to consider for commercial deployment, as it would require fine-tuning the system with data from the actual end-user. Fourth, we quantitatively measured the effect of such variability for all types of signals, demonstrating its huge relevance, and shown that deep learning models dedicated to time series processing are better able to generalize across users than the more commonly employed shallow machine learning models. Finally, with a focus on in-vehicle deployability, which is of significant industrial interest, we measured also such metrics as model size, inference time, and energy consumption, showing feasibility on two embedded platforms, which is a key advancement towards on-board deployment of robust, real-time cognitive distraction detection systems.