Real-time Multi-object Tracking Research Based on Image and Point Cloud Fusion

Negotiating an optimal balance between accuracy and real-time in object detection and tracking technologies remains a key challenge within the autonomous driving realm. This study presents IAFusionMOT, a real-time multi-object tracking algorithm that fuses image and point cloud data, resolving the limitations of unimodal tracking and the complexities of multi-object tracking. IAFusionMOT incorporates a novel four-level associated structure that synergistically integrates the combined detection outputs with the trajectory library, enabling adaptability to various environments. It employs geometric data from the 3D detection frames to formulate an efficient cost function that optimizes the association between sequential frames using Hungarian matching. This study benchmarks IAFusionMOT on the KITTI tracking dataset, where it significantly outperforms existing trackers. Particularly in pedestrian tracking, the Higher Order Tracking Accuracy(HOTA) metrics reaches 51.12%, an improvement of 12.5% and the single-frame inference speed increased to 164 FPS, demonstrating the algorithm’s superior performance in both accuracy and speed. Further field-testing in real vehicles confirms the algorithm’s robust ability to track objects in diverse scenarios.