Design Flow of mmWave Radar and Machine Vision Fusion for Pedestrian Collision Warning

This paper presents a millimeter wave (mmWave) radar and Machine Vision fusion system to alert drivers of potential pedestrian collisions. The system is composed of two subsystems, the mmWave Pedestrian Localization subsystem and the Machine Vision Pedestrian Classification subsystem. The mmWave Pedestrian Localization subsystem obtains the relative location of the pedestrians using a mmWave radar sensor while the Machine Vision Pedestrian Classification subsystem uses Histogram of Oriented Gradients and Support Vector Machine algorithms to classify pedestrians in a camera’s field-of-view. The two-layer pedestrian detection design protects the system from any miss detections within a single subsystem. By utilizing mmWave technology with Machine Vision, the safety operation of cars and pedestrian safety can be increased. The proposed system utilizes Texas Instrument’s AWR1642BOOST mmWave Radar with the high computing power of NVIDIA’s Jetson Nano