Robust Vehicle Localization for Spherical Camera Models: Solution, Framework, and Verification

Vehicle visual localization uses vision sensors to capture environmental information, enabling precise localization of autonomous vehicles within their surroundings. However, current visual localization methods generally have some shortcomings: on one hand, they are limited by the camera’s field of view, on the other hand, their robustness is often inadequate under challenging conditions such as lighting changes, long-term scene changes, or occlusions. To address these issues, we formulate a general spherical camera model for both fisheye and panoramic cameras and propose a minimal solution for pose estimation using this model based on vehicle motion characteristic. The minimal solution cannot filter outliers, so a robust estimation framework is necessary. For outlier-rejection, we introduce two frameworks: a probabilistic optimal RANSAC and a globally optimal graph-based framework. We conduct a probabilistic analysis of the RANSAC to demonstrate its enhanced robustness given by the proposed minimal solution. To achieve robustness to extreme outliers (higher than 90%), we decouple the rotation and translation space through the minimal solution to construct maximum consensus graph for the two sub-problems. We then employs a maximum clique search algorithm to find the optimal solutions, achieving deterministic convergence while maintaining real-time performance. Extensive experiments with synthetic data, real-world fisheye images, and 360° panoramic images validate the robustness and efficiency of our proposed algorithms.