Multimodal Remote Sensing Object Detection Based on Prior-Enhanced Mixture-of-Experts Fusion Network

Multimodal remote sensing image object detection enhances detection accuracy by fusing complementary information from multimodal image data. However, complex environments significantly affect the reliability and complementarity of multimodal data, and traditional methods struggle to dynamically adapt to environmental changes, leading to degraded detection performance. To address this challenge, in this article, we propose a multimodal remote sensing object detection method based on a prior information-enhanced mixture-of-experts fusion network (PMFN). Specifically, we first introduce a PMFN framework to achieve environment-adaptive multimodal image feature fusion. Second, we propose a dynamic gating network that combines prior information and multimodal image features to endow the system with environmental perception capabilities. This network is employed to dynamically allocate weights to subfusion experts optimized for different environmental conditions within the mixture-of-experts (MoE) fusion network framework. Furthermore, to fully exploit the complementary information present in multimodal image features, we propose a frequency-decoupled feature fusion network as a subfusion expert within the MoE fusion network framework. This utilizes wavelet transform to decouple the features of each modality and then develops personalized fusion strategies for each frequency subband. In addition, to enhance detection efficiency, we introduce a cross-scale feature channel interleaved fusion (CSFCIF) strategy, which significantly reduces computational cost while ensuring stable detection performance. Experimental results on the DroneVehicle and RGBT-Tiny datasets demonstrate that our method achieves competitive performance compared to state-of-the-art approaches. Code will be available at: https://github.com/LiuKewei0110/MDPMFN