Pedestrian Liveness Detection Based on mmWave Radar and Camera Fusion

Autonomous driving requires vehicles to achieve fine detection of objects in the surrounding environment, especially living pedestrians. Nevertheless, in real world road environments there are living pedestrians and roadside portrait billboards. Existing vision-based object detection technologies fail to ac-curately distinguish living pedestrians from human figures. As an important sensor of autonomous driving system, mmWave radar has extra help to detect living pedestrians. In this paper, we extract the radar cross section (RCS) of the object from the low-cost mmWave radar signal as a distinguishing feature between living pedestrian and portrait billboard. Based on this observation, we propose a feature fusion network of mmWave radar and computer vision based on attention mechanism, and detect living pedestrians from fusion features. We implement the design with commodity mmWave radar IWR6843ISK-ODS and RGB camera Logitech Pro C920. The evaluation results show that our method effectively detects living pedestrians with an mAP of 97.7% and outperforms existing studies.