Localization Based on Semantic Map and Visual Inertial Odometry

Autonomous vehicles require precise localization for safe control. This paper presents a localization approach based on semantic map and visual inertial odometry for autonomous vehicles. Our approach uses consumer grade parts, and only relies on a single front camera and a consumer grade IMU and a GPS. Using real-time semantic landmark detection and real-time visual inertial odometry, we localize the full 6-DOF pose of the vehicle in the semantic map with mean absolute accuracy at less than 20 cm, With this accuracy, we can achieve high levels of autonomy, and speed up the evolution of autonomous driving. The main contributions of our approach are: (i) 2D-3D semantic landmark matching in continuous frames; (ii) full 6-DOF pose optimization with semantic constraints in a sliding time window.