NeRFMatching for Vehicle Localization: Cross-View Matching Through Domain Adaptation With NeRF Representations

Cross-view matching technique for aerial and terrestrial images provides essential global pose constraints for terrestrial vehicle systems, aiding in the reduction of trajectory drift over long distances and extended periods. Existing research primarily focuses on enhancing the effectiveness of cross-view image matching through feature learning or geometric transformations. However, these methods are limited by the information available in single images, making it difficult to fundamentally address the discrepancies between images captured from different viewpoints. This article introduces the NeRFMatching framework, an innovative approach using aerial-terrestrial image nural radiance fields (NeRFs) inversion to effectively address the matching challenges presented by large viewpoint differences. The central idea of this method is to use multiview aerial imagery to construct a NeRF model, employing it as a bridge to establish a matching relationship between terrestrial images and original aerial images, thereby recovering the pose information of the vehicle platform. Specifically, to bridge the vast cross-view domain gap, this study proposes a pose-guided NeRF image projection method based on the relative pose information of the original aerial images. To validate the effectiveness of this approach, ISPRS benchmark dataset and self-collected dataset were used in experimental validation. The experimental results demonstrate that the NeRFMatching method effectively addresses the challenges of cross-view aerial-to-ground matching and precise localization, achieving meter-level accuracy. This study compared NeRFMatching with three state-of-the-art visual localization algorithms. The findings indicate that NeRFMatching consistently achieves the smallest localization errors across most scenarios, effectively enhancing localization precision from the 10-m level to the meter level.