Sea Surface Multi-Target Tracking Based on Fusion of LiDAR and Panoramic Camera

This paper presents a sea surface target tracking algorithm based on the fusion of LiDAR point clouds and panoramic images, suitable for short-range target perception during autonomous navigation of Unmanned Surface Vehicle (USV). The image-based target tracking algorithm relies on the correlation and matching between the detected target and the existing track. However, the motion state of the target perceived by the USV is always relative when it is sailing autonomously. Consequently, the target's position and contour information perceived by the USV can change drastically between frames, potentially leading to incorrect target association and tracking errors. To address the aforementioned issues, the target's motion state is converted from relative to absolute when tracking in the LiDAR point cloud and panoramic image, and the reliance on contour information is minimized as much as possible. The effectiveness of the proposed algorithm is verified by the autonomous navigation experiment of USV, and compared with the target tracking algorithm based on single sensor design.