Detecting Drivers' Mirror-Checking Actions and Its Application to Maneuver and Secondary Task Recognition

This study explores the feasibility of detecting drivers' mirror-checking actions using noninvasive sensors. Checking the mirrors is an important primary driving action that allows drivers to maintain their situational awareness, particularly when they are planning to turn or change lanes. Recognizing when drivers are checking the mirrors can facilitate the detection of hazard scenarios by considering contextual information (e.g., turning without checking mirrors, lack of mirror-checking actions signaling cognitive distractions, or distinction between gazes due to primary or secondary tasks). This study analyzes drivers' mirror-checking actions under various real driving conditions. We analyze the drivers' mirror-checking actions under normal conditions, as well as when the drivers are engaged in secondary tasks such as tuning the radio or operating a cell phone. We also compare mirror-checking behaviors observed during different maneuver actions: driving straight, turning, and switching lanes. This study reveals statistically significant differences in mirror-checking actions among most of the comparisons. The results suggest that mirror-checking actions can be useful indicators in recognizing drivers engaged in secondary tasks, as well as in detecting driving maneuvers. We propose to detect mirror-checking actions using features extracted from multiple noninvasive sensors (CAN-Bus and cameras facing the driver and the road). We consider three machine learning algorithms for unbalanced data sets, achieving an F-score of 91%. The recognized mirror-checking actions are used as additional features to improve the performance of secondary task detection and maneuver recognition. These promising results suggest that it is possible to detect mirror-checking actions, providing contextual information to improve new driver monitoring systems.