Accuracy- and Simplicity-Oriented Self-Calibration Approach for In-Vehicle GNSS/INS/Vision System With Observability Analysis

With significant advances in manufacturing technology, multiple sensors such as global navigation satellite system (GNSS) devices, inertial measurement units (IMU) and cameras have become affordable in intelligent vehicle systems and are generally integrated for motion estimation and environmental perception. In such integrated systems, accurate and flexible sensor calibration is a basic foundation for optimal information fusion. However, traditional calibration methods are generally labor-intensive or only applicable to handheld platforms. To address this issue, we propose an accurate and plug-and-play self-calibration method for in-vehicle GNSS/inertial navigation system (INS)/Vision system. Without relying on specially-designed targets and initial guess of sensor setups, all involved sensing parameters, including GNSS-IMU and Camera-IMU spatial-temporal extrinsics, are properly initialized and estimated online via a centralized Extended Kalman Filter (EKF). Besides, an observability constrained module is also developed to address degradation cases through detailed observability analysis. The results in both Monte-Carlo simulations and real-world experiments indicate that the proposed method enables accurate online calibration of all spatiotemporal parameters involved in GNSS/INS/Vision systems and shows superior calibration performance over the state-of-the-art methods. Furthermore, for the first time, we prove the superiority of using raw GNSS observations to enhance camera-IMU calibration than using processed GNSS positions or not using any GNSS information.