Holistic Driver Monitoring: A Multi-Task Approach for In-Cabin Driver Attention Evaluation through Multi-Camera Data

In the domain of road safety, evaluating and enhancing driver attention is crucial for reducing the frequency of road accidents. This research paper proposes a novel methodology for assessing driver attention by leveraging multi-camera data. Our approach considers a dual-camera setup capturing the face and body parts from the driver’s side, focusing on four key tasks: distraction detection, gaze direction analysis, fatigue detection, and hands-on-wheel monitoring. We utilize facial landmarks for fatigue detection, specifically targeting the mouth and eye regions. SqueezeNet, a lightweight convolutional neural network, is employed to discern signs of driver fatigue. Gaze direction analysis uses the same network but focuses solely on eye landmark features. Distraction activities are identified by extracting optical flow features and using them as input to the robust I3D network. Further, hands-on-wheel detection is achieved by extracting hand landmarks, followed by using a 3D CNN model. We propose a driver attention score to consolidate these tasks into a unified measure of driver attention. This score is a holistic representation of the driver’s attentiveness, combining insights from distraction detection, gaze direction analysis, fatigue detection, and hands-on-wheel monitoring. Our methodology is validated on a driver monitoring dataset (DMD), where training and application demonstrate the effectiveness of the proposed approach in assessing and quantifying driver attention.