Driver Activity Recognition Using Deep Learning and Human Pose Estimation

Driver Monitoring Systems play a crucial role at detecting driver’s distraction to assure perception and reaction when driver is loaded by secondary activities, and specifically making transition of control authority from automated driving system to driver in conditional automation (SAE Level L3). Activities such as using cell phone, operating radio, drinking, talking to a passenger are an instance preventing the driver percepting the road scene and, degrading perception and reaction of the driver to possible hazards. In this paper, a new approach is presented to detect driver activity using deep learning based driver image classification and driver pose estimation. Driver pose information is obtained using a pre-trained deep network and it is used in a machine learning based classifier, in this case, a Random Forest. Then, results of two predictive models, deep learning based image classifier and Random Forest classifier leveraged by pose estimation, are combined and activity detection is performed using this fusion mechanism. The effectiveness of the presented driver activity recognition system is investigated using a publicly available dataset and multi-class activity detection test accuracy is found to be 0.9703.