A Failure-Resistant, Lightweight, and Tightly Coupled GNSS/INS/Vision Vehicle Integration for Complex Urban Environments

Accurate ego-motion estimation is the foundation for autonomous vehicle navigation in complex urban environments. In multisensor fusion schemes, in addition to environment-related degradations, sensor measurements sometimes fail, which may have an unpredictable impact on navigation performance. To make the vehicle sensor system resistant to different types of failures, we propose a failure-resistant and lightweight multisensor fusion method, in which pseudorange and carrier phase from multiple global navigation satellite system (GNSS) constellations, sparse visual features from a monocular camera, and records from multiple low-cost micro-electromechanical system (MEMS) inertial measurement units (IMUs) are integrated at the measurement level by a sliding-window tightly coupled filter. Furthermore, considering the possibilities of the high-rate IMUs recovering from failures, we propose a flexible strategy to fully utilize all available IMU measurements without frequent reinitializations. To verify the proposed method, we perform extensive evaluations under different sensor failures (e.g., GNSS short-term blocking and long-term failures, image failures, and complex IMU failures and recoveries). The results show that our method outperforms conventional approaches that include only a single IMU and merely focus on sensor degradation, and resists various types of sensor failures. Despite initializing under a harsh environment and experiencing complex sensor failures, velocity and yaw estimations are significantly improved while a submeter 3-D positioning can be achieved in approximately 80% of the challenging scenario. For complex IMU failure and recovery records, our method can fully utilize all available IMU measurements and seamlessly switch between the base and auxiliary IMUs to recover trajectories that approximate the ground truth.