Direct Aerial Visual Localization using Panoramic Synthetic Images and Domain Adaptation

In the realm of aerial vehicle navigation, the reliance on satellite-based and external localization methods presents vulnerabilities to various interferences. This drives the necessity for a self-sufficient absolute navigational system. Image-based localization methods, particularly Absolute Visual Localization (AVL), directly determine the pose in the global frame from a given image. A workflow using 360-degree panoramic images for image-based localization, driven by a Deep Convolutional Neural Network (DCNN), is proposed. Utilizing panoramic imagery offers the advantage of encompassing visual information from all angles. Synthetic data generated from multiple sources such as photogrammetry, Open Street Map (OSM), and official 3D building data are used to train the localization network. Domain adaptation using cycleGAN is also used to bridge the Sim2Real gap and enhance model performance. Utilizing OSM features are shown to improve localization performance (median Euclidean error) by at least 13%, and a further 20% with cycleGAN dataset augmentation. Closed loop control is also achieved using a trained model, enabling a quadrotor prototype to hover within a 1 m circle.