Viewpoint Invariant 3D Driver Body Pose-Based Activity Recognition

Driver monitoring will be required in many countries for all new vehicles with automation functions. While the common approach for this task is face and eye gaze monitoring, cameras with a wider field of view have already been introduced in some cars. However, their mounting position can change between vehicle models. To minimize data collection efforts and to facilitate data reuse it is important for algorithms to be able to deal with a changing environment. We conduct an experiment comparing the performance of video-based models with 3D body pose-based activity recognition methods with regards to sensor and position changes. We introduce a modular activity recognition pipeline which uses a sensor independent representation including the 3D body pose of the driver, 3D dynamic object positions and 3D interior positions. We show that while video-based models offer the best quality overall, when trained and tested on the same camera view, body pose-based methods can be far more robust to positional changes. Moreover, augmentation reduces the performance drop across views to 35% compared to 83% without augmentation and 90% for video-based models.