Semantic Intrinsic and Extrinsic LiDAR-Camera Calibration in Targetless On-Orbit Environments

With the increasing complexity of on-orbit service tasks, there is a growing demand for multi-modal sensing capabilities. LiDAR and camera are the primary sensors for space service robotics, their data fusion can enhance perceptual understanding. However, the intrinsic and extrinsic parameters of LiDAR and camera may be affected by temperature and vibration during launch. How to re-calibrate the intrinsic and extrinsic parameters in a targetless on-orbit environment remains an open problem. In this paper, we propose a coarse-to-fine intrinsic and extrinsic calibration method that utilizes line and plane semantic features extracted from satellite panels in sequential data to achieve joint calibration. Initially, we optimize the intrinsic and extrinsic parameters roughly based on reprojection errors derived from line features. Subsequently, fine optimization is performed using plane constraints obtained from SfM point cloud aligned with LiDAR point cloud in each frame, along with 3D-2D line feature matching between LiDAR registration point cloud and image in each frame. The method was validated on a simulated dataset consisting of satellite fly-around image and point cloud sequences within the range of 10 to 20 meters. The results show that the average rotation error is 0.24°, the average translation error is 0.02 m, and the average intrinsic error is 1.33 pixels, which proves the accuracy and robustness of the proposed method.