A Coarse to Fine Indoor Visual Localization Method Using Environmental Semantic Information

In this paper, we focus on the camera localization problem using visual semantic information. In contrast to the state of the artworks which often use visual features to do localization, we here propose a coarse to a fine mechanism to localize the camera position. First, a semantic database including object information around the target environment is constructed using a deep learning method. Second, for the coarse step of the visual localization, we match class attributes of objects in the current frame to the object database and find candidate frames that have similar objects. Third, the most similar candidate frame to the current frame is selected by CNN features. For the fine step of localization, the final pose of the camera can be estimated using feature matching with semantic information. Compared to the state of the art visual localization methods, the proposed localization method based on semantic information has higher localization accuracy. Furthermore, the proposed framework is not only useful for visual localization, but also useful for other advanced tasks of robot, e.g., loop closing detection, object searching, and task reasoning.