Hybrid visual and inertial position and orientation estimation based on known urban 3D models

More and more pedestrians own devices (as a smartphone) that integrate a wide array of low-cost sensors (camera, IMU, magnetometer and GNSS receiver). GNSS is usually used for pedestrian localization in urban environment, but signal suffers of an inaccuracy of several meters. In order to have a more accurate localization and improve pedestrian navigation and urban mobility, we present a method for city-scale localization with a handheld device. Our central idea is to estimate the 3D location and 3D orientation of the phone camera based on the knowledge of the street furnitures, which have a high repeatability and a large coverage area in the city. Firstly, the use of inertial measurements acquired with an IMU in the vision based method allows to accelerate the calculation of the position and orientation. Secondly, the weighted fusion between the rotation matrices calculated with the vision and the inertial processes allows to give the more importance in the calculation with the highest confidence. With a good points selection, this provides a localization that is in the GNSS post-processed measurement precision use for determining the position and the orientation of the street furnitures. Performances are presented in terms of accuracy of positionning. The final aim is to have with our method a precision good enough to be able to propose in future works a on site display in augmented reality.