3D object tracking using RGB and LIDAR data

Object tracking is one of the key components of the perception system of autonomous cars and ADASs. With tracking, an ego-vehicle can make a prediction about the location of surrounding objects in the next time epoch and plan for next actions. Object tracking algorithms typically rely on sensory data (from RGB cameras or LIDAR). In fact, the integration of 2D-RGB camera images and 3D-LIDAR data can provide some distinct benefits. This paper proposes a 3D object tracking algorithm using a 3D-LIDAR, an RGB camera and INS (GPS/IMU) sensors data by analyzing sequential 2D-RGB, 3D point-cloud, and the ego-vehicle's localization data and outputs the trajectory of the tracked object, an estimation of its current velocity, and its predicted location in the 3D world coordinate system in the next time-step. Tracking starts with a known initial 3D bounding box for the object. Two parallel mean-shift algorithms are applied for object detection and localization in the 2D image and 3D point-cloud, followed by a robust 2D/3D Kalman filter based fusion and tracking. Reported results, from both quantitative and qualitative experiments using the KITTI database demonstrate the applicability and efficiency of the proposed approach in driving environments.