The Visual Localization Based On The Semantic Error Image

The Outdoor visual localization can get the position (the spatial location and direction) of the query target through semantic information, three-dimensional point cloud, images, geometric information, and other data [1]. It has been widely used in robot Localization and navigation, autonomous vehicle navigation, augmented reality, 3D reconstruction, SLAM (Simultaneous Localization and Mapping), etc. At present, the most advanced visual localization method is still the traditional localization based on three-dimensional, which has the best localization accuracy. However, due to the 2D-3D matching mechanism, the accuracy of it is seriously reduced in challenging scenes with great scene changes like viewpoint variation, illumination variation. For the problem, in this article, we come up with a model based on the semantic error image which is based on the semantic information of 2D images and 3D point clouds. Based on the original structure-based localization method, we introduce the semantic error image, we use the semantic data of the image sequence to construct the semantic error image based on the semantic information of 2D and 3D points, and utilize these information as a correction signal to eliminate the wrong match and improve the accuracy of localization. We test and compare these methods on two challenging datasets with large scene changes, the experimental results showed that our well method based on semantic error image was significantly better than the method based on 3D structure on the challenging datasets, and the localization accuracy was significantly improved.