Accurate and Consistent Spatiotemporal Calibration for Heterogenous-Camera/IMU/LiDAR System Based on Continuous-Time Batch Estimation

Nowadays, multisensor fusion, which could compensate for the deficiencies of stand-alone sensors, has been widely accepted as a feasible solution for navigation and perception tasks. To achieve optimal information fusion, accurate intersensor spatiotemporal transformation is a fundamental prerequisite. However, few existing calibration methods could accurately and flexibly calibrate spatiotemporal parameters for camera/inertial measurement unit (IMU)/light detection and ranging (LiDAR) sensor suites without relying on artificial targets and manual intervention, especially for those low-cost suites that employ rolling shutter (RS) cameras. To this end, building upon our previous work (i.e., two-step LiDAR-IMU-camera (LIC)-calib), we propose a spatiotemporal calibration method for heterogenous-camera/IMU/LiDAR systems, which supports both global shutter (GS) and RS cameras, and enables intrinsic refinement. In particular, a novel initialization method is developed to provide accurate initial guesses, requiring no prior knowledge or calibration infrastructure. To further improve the calibration consistency and accuracy, the split calibration of camera/IMU and LiDAR/IMU in two-step is unified into a global nonlinear optimization problem. Both simulated and real-world experiments were conducted for quantitative evaluation, and the results demonstrate that the proposed method outperforms its predecessor and other state-of-the-art methods.