A Cross-Modal Registration Method for Visible and Infrared Images of Small UAV Targets

Cross-modal unmanned aerial vehicle (UAV) target detection and recognition has shown great advantages under complex background, such as poor illumination, shading and deformation, etc., because cross-modal data can provide complementary information mutually. However, the real cross-modal or multi-modal data suffers from the problem of position shift and spatial deformation, that is, the image pairs are not strictly aligned, which is extremely unfriendly to downstream detection, recognition or tracking tasks. In this paper, we propose a cross-modal registration method for visible and infrared images of small UAV. Firstly, a coarse registration module based on ROI coordinates is designed to perform local registration of unregistered images, which can fully extract and utilize the small-scale feature information in the cross-modal images. Then, a semantic-based fine registration module is further proposed for fine correction and matching. In addition, we manually rearrange and relabel the Anti-UAV dataset, and experimental validations on the dataset are performed, demonstrating the effectiveness of the proposed algorithm by comparing with classical approaches.