Tightly Coupled Integration of LiDAR and Vision for 3D Multiobject Tracking

Three-dimensional (3D) multiobject tracking (MOT) plays a pivotal role in emerging intelligent transportation system (ITS) applications, such as intelligent vehicles, wherein it provides accurate and reliable motion information of surrounding objects for safe navigation and motion planning. However, vision-based approaches are susceptible to occlusions and LiDAR-based methods struggle with object identification due to the lack of texture information, posing challenges in object tracking. To overcome the inherent limitations of these individual sensors, we propose a fusion-based 3D MOT system that tightly couples LiDAR and vision measurements. Specifically, we first design an aggregated association metric that fuses the detections and geometric features from both LiDAR and image domains, to effectively track objects. On this basis, all associated multimodal measurements are further integrated into a unified factor graph optimization framework to refine the tracked 2D and 3D trajectories. The extensive experiments conducted on public datasets demonstrate that our system significantly outperforms the state-of-the-art fusion-based MOT methods in terms of both object localization accuracy and MOT performance.