A Deep Learning Approach for In-Vehicle Multi-Occupant Detection and Classification using mmWave Radar

Due to several benefits including a wide field of view, fine resolution, and low cost, millimeter-wave (mmWave) radars are of high interest for in-vehicle sensing tasks, including occupant detection and classification. While general presence detection, which identifies any living presence across all seats, is very accurate using model-based methods, limited angular resolution, multipath reflections, and ambient reflections impede localized seat-by-seat detection and classification of occupants. In this paper, we propose a novel deep learning solution using 3D point clouds obtained from a mmWave radar mounted in-cabin. By focusing on sparse 3D point clouds rather than fully populated range-angle heatmaps, we obtain a 54.1% reduction in computational complexity and a 94.7% reduction in data storage requirements while preserving estimated velocity per point. Furthermore, our method addresses challenges due to ambient and multipath reflections in the vehicle by constraining the spatial focus of our model. Evaluations demonstrate 95.6% accuracy for localized detection and 88.7% accuracy for classifying the occupant as adult, child, or baby when testing on participants unseen during training and validation.