Online Spatial and Temporal Initialization for a Monocular Visual-Inertial-LiDAR System

Multisensor fusion of visual-inertial-LiDAR systems (VILSs) has become a research hotspot in recent years. Most VILSs require offline calibration of external parameters and hardware time synchronization to ensure successful initialization of the system. However, such requirements are not easy to achieve in practice. We propose an online initialization method for VILSs that can achieve software time synchronization (temporal) and automatically calculate external parameters (spatial) without prior knowledge of environmental information or special movements. Our approach consists of two phases: (1) novel monocular visual inertial (VI) initialization and (2) LiDAR-added (LI) online initialization. In phase 1, our method considers more complete initial states, including the time offsets, biases of the accelerometer, and translation between the inertial measurement unit (IMU) and camera. We implement faster and more robust VI initialization than VINS-MONO. In phase 2, we linearly interpolate the visual-inertial odometry (VIO) onto the same timestamps of the LiDAR and solve the LI external parameters by aligning the VIO with LiDAR odometry (LO). Our LI initialization phase is more practical due to the current lack of an online LI initialization method. Additionally, to eliminate the bundling effect of the initial state estimation in the two phases, we provide a linear solution in nominal-state space to obtain a rough starting value of the rotation extrinsic parameter first. Then, nonlinear optimization under motion constraints is introduced to obtain the full initial states in true-state space. The performance of the proposed method is verified on both a public dataset and a self-assembled handheld VILS device in real-world experiments.