A Generic LiDAR-Camera Extrinsic Calibration Method base on Lightweight SAM

In the domains of autonomous driving and robotic vision, camera and LIDAR, as core sensors, played an imperative roles where the precise fusion of their data had a significant impact on environmental perception. Accordingly, the accurate calibration of external parameters to achieve data fusion emerged as a crucial step to enhance system performance. This paper proposed a zero-shot, target-less and rapid method for calibrating the external parameter of the LiDAR-Camera, based on the distillation of the large model, Segment Anything Model (SAM). Initially, the SAM model was distilled, resulting in a lightweight image segmentation model that could rapidly segment objects in any scene and obtain a mask for each object. Subsequently, a process for handling LiDAR point cloud data was designed, which could obtain the attributes of each point cloud and adaptively acquire the weights of each attribute based on the scene. After processing the image and point cloud data, the external parameter from the LiDAR to the camera was optimized by maximizing the consistency of the point cloud projection to the segmented image mask. The proposed method was evaluated on numerous public datasets and self-built datasets, and compared with other advanced methods. The results indicated that our method exhibited higher accuracy and speed in different scenes.