Multi-Modal Tracking Using LiDAR and Visual Signals

Multi-object tracking (MOT) is a crucial technique for detecting and tracking multiple objects over time in a scene. It involves locating objects in consecutive frames of a video or sequential observations and establishing correspondences to maintain their identity. MOT is essential in various applications, including autonomous vehicles, robotics, and video surveillance. In autonomous vehicles, it is typically performed using 3D models (LiDAR/stereo) or 2D models (one or more RGB cameras). The LiDAR 3D point-cloud provides a 360-degree view of its surroundings but has a low resolution compared to the camera's visual signals. This paper presents a sensor fusion approach for multi-object detection and tracking, combining object detection information from sensors, LiDAR, and cameras and applying data association algorithms like Hungarian algorithms for tracking to increase the overall system. The EagerMOT approach for 3D Multi-Object Tracking via Sensor Fusion achieves state-of-the-art results across several MOT tasks on the KITTI dataset, achieving an accuracy of 87.95 %.