A Robust Multiobject Tracking Method Based on 4-D Millimeter-Wave Radar and Monocular Vision Fusion

Multiobject tracking (MOT) is an important component of the perception system for autonomous vehicles and mobile robots. The 4-D millimeter-wave (MMW) radar is a novel and promising sensor for MOT for its capability to provide 3-D position data and additional Doppler data (
$\text {3-D} + \text {1-D}$
). However, the MOT methods using 4-D MMW radar remain scarce, and existing research commonly disregards the potential simplicity of MOT. Moreover, the application of 4-D MMW in MOT is constrained by the noise and sparsity of the radar point clouds. To address the above problems, we propose a robust MOT method with a simple modular structure, which integrates 4-D MMW radar and monocular vision. The radar data and vision data are first processed in parallel to obtain object detections separately. The detections of the two sensors are then matched and fused using a proposed association method in the camera domain, which can avoid association ambiguities. Finally, a composite structure, mainly consisting of a detection-to-trajectory association module named Fusion-BYTE and a trajectory management module, is designed to exploit the processed sensor detections and yield tracking results. The proposed method is extensively verified on the public view-of-delft (VoD) dataset and our unmanned aerial vehicle (UAV) dataset. The experimental results demonstrate that our method achieves superior tracking performance compared to the state-of-the-art single-sensor and multisensor fusion MOT algorithms while maintaining high operational efficiency.