Cross-modal Adaptive Fusion Object Detection Based on Illumination-Awareness

In recent years, cross-modal object detection has attracted much attention from researchers across various domains. Compared to single-modal detection, cross-modal object detection combines diverse features from distinct modalities, bolstering the reliability and robustness of object detection applications. However, presently proposed cross-modal models exhibit deficiencies in terms of fusion methodologies, resulting them less suitable for the specific task of aerial object detection.To address these challenges, we present the Cross-Modal Adaptive Fusion Network based on Illumination Awareness (CAFN-IA). The Illumination Awareness module is designed to dynamically adjust the position and orientation of GroundTruth-Box (GT-Box) by quantifying light intensity and computing Intersection over Union (IoU) metrics derived from RGB and Infrared images. Additionally, a dual-stream network architecture is developed to extract RGB and Infrared features separately. Moreover, the introduction of an Interest Region Extraction module enhances the extraction of partial regions. Furthermore, we introduce a Cross-Scale Adaptive Fusion module, enhancing the complementarity of distinct features generating from RGB and Infrared images. Notably, our approach involves the modification of the loss function to elevate the accuracy of small object detection.Extensive experimentation and thorough ablation studies demonstrate the efficacy of our method, yielding an accuracy rate surpassing 69% on the DroneVehicle Datasets.