C2DFF-Net for Object Detection in Multimodal Remote Sensing Images

Multimodal fusion detection has proven to be a practical approach for improving small target perception in remote sensing. Existing studies primarily focus on the fusion of multimodal spatial-domain features, while paying insufficient attention to cross-domain differential features (DFs). Furthermore, exploration of lightweight fusion mechanisms tailored for deployment on edge devices such as UAVs remains relatively limited. To address these issues, we propose an extremely lightweight cross-modal and cross-domain DF fusion network (C²DFF-Net). The network includes three innovative plug-and-play modules. Specifically, we propose a cross-modal DF interaction module (CDFIM) to facilitate intermodal information interaction by enhancing DFs between visible and infrared modalities. To improve the adaptive fusion of multimodal complementary features, we propose a cross-domain gated self-attention (CGSA) module. The global context information of the image is explored from a frequency-domain perspective, and a polarized self-attention (PSA) mechanism is introduced to establish long-range dependencies on spatial-frequency-domain DFs while filtering redundant information. In addition, we design an adaptive light-aware mask (ALM) module to enable the network to learn effective multimodal complementary features without bias, improving its robustness in complex lighting environments such as exposure and glare. Comprehensive experiments demonstrate that C²DFF-Net not only achieves state-of-the-art (SOTA) performance but also maintains a lightweight design, achieving 85.7% mAP50 with 6.58 M parameters and 14.6 giga floating-point operations (GFLOPs) on the DroneVehicle dataset. Furthermore, it shows a favorable accuracy-computation tradeoff compared to SOTA models on the VEDAI and FLIR datasets. Finally, we conduct real-world deployment experiments, and C²DFF-Net achieves satisfactory results under various lighting conditions, clearly demonstrating its practical applicability. Code will be available at https://github.com/FPGAzzy/C2DFF-Net.