Fast Detection of Handheld Phone-Distracted Driving by Sensing the Driver's Hand-Grip

Handheld phone distraction is the leading cause of traffic accidents. However, few efforts have been devoted to detecting when phone distraction happens, which is a critical input for taking immediate safety measures, not only related to the driver but also important to surrounding vehicles, pedestrians, and vehicle networks. This work proposes a fine-grained handheld phone-use monitoring system, which detects the start of phone-distracted driving activities and further distinguishes different handheld phone-use scenarios, which enables estimating the impacts on traffic safety to take measures accordingly. Specifically, the proposed system emits periodic ultrasonic pulses to sense if the phone is being held in hand or placed on support surfaces (e.g., seat and cup holder), and the unique signal interference resulted from the contact object's damping, reflection, and refraction is analyzed based on the sounds that return to the microphone. We derive the short-time Fourier transform to describe such impacts and develop a CNN-based binary classifier to distinguish phone use between handheld and handsfree. Moreover, the system leverages the embedded inertial sensors to capture the phone's motion dynamics and recognize specific handheld phone distractions (e.g., holding the phone for calling). The system periodically samples the driver's phone-use status and use an error correction window to correct misclassified samples. As a result, the start, end, and duration of each handheld phone distraction activity can be obtained. Extensive experiments show that our system achieves 99.5% accuracy in recognizing handheld phone-use instances and a 0.76-second median error in detecting the start of a handheld phone distraction.