Vehicle Localization in Six Degrees of Freedom for Augmented Reality

Augmented reality visualizes map content or sensor observations by projections that fit naturally into the surroundings-thereby providing relevant information in a convenient and safe way for the customer. A stable projection of map data into the driver's field of view requires precise knowledge of the vehicle's pose in six degrees of freedom (dof) with respect to a map. We developed an efficient 6dof localization based on odometry, camera recognized landmarks and a high definition (HD) map using an extended Kalman filter (EKF). As a result, we compared the accuracy of two fusion methods on simulated driving scenarios based on real HD map data: (i) fusing 2D landmarks from the image plane directly, and (ii) fusing 3D landmarks assuming prior 3D reconstruction. Using five types of traffic signs and pole-like landmarks, we found that the accuracy difference between the approaches reduces with increasing number of landmarks per cycle. In presence of five or more landmarks per cycle, the 2D measurement approach reaches the accuracy of the 3D measurement approach. Thus, with many landmark types, the 2D approach becomes a beneficial alternative: While achieving comparable accuracy, it circumvents 3D reconstruction, thereby reducing the computational effort, which is highly desirable in autonomous driving.