Direct alignment with generalized correspondences: A unified framework for structure-based visual pose estimation

Visual pose estimation is a fundamental problem for autonomous robot navigation. Basically, methods for structure-based visual pose estimation can be categorized into direct and indirect based methods considering the way of data association. Indirect based methods extract features to establish correspondences across images, while direct based methods align images under the assumption of local consistency. In this paper, we propose a unified framework for visual pose estimation to show that typical methods within these two categories, including photometric and edge-based alignment, feature-based matching methods can all be unified for visual pose estimation, among which the differences only lie in the design of a multi-layer scalar field. Upon the framework, we derive an optimal scalar field for visual pose estimation, generalizing the data association to point-to-region correspondence. With the theoretic analysis as reference, we discuss their performances in terms of convergence basin, data association and outlier robustness, and furthermore propose a general visual pose estimator that integrates the insights from direct and indirect based methods. In experiments, the combined estimator outperforms the other methods with one kind of data association, validating the effectiveness of the framework as well as the correctness of the theoretic analysis.