Vision-Based Pose Optimization Using Learned Metrics

Camera pose optimization is the basis of geometric vision works, such as 3D reconstruction, structure from motion, and visual odometry. A pose optimization method based on learned metrics is proposed to improve the optimization convexity. The neural network was designed and trained based on the collected datasets, respectively. The network inputs pairwise patches and outputs the Euclidean distance of its center. This distance is involved in the residual calculation of Gauss-Newton, and the Jacobian corresponding to this distance can be analytically solved. The simulation verified convergence and generalization of the designed network. The accuracy and robustness of the proposed pose optimization compared with intensity- and feature-based optimizations are also verified.