HF-MCD: A Heterogeneous Fusion Framework for Multimodal Change Detection

Multimodal change detection (MCD) aims to detect changed areas between the bi-temporal multimodal images, such as the RGB, panchromatic (PAN), multispectral (MS), and synthetic aperture radar (SAR) images, which has attracted attention in recent years. However, existing deep learning-based methods for MCD tasks still face several heterogeneity factors, the first one is the spatial resolution differences in multimodal data, which leads to the semantic gap between multimodal features. To solve this problem, we propose the heterogeneous collaborative fusion (HCF) module to integrate the multimodal features with spatial gaps. The other one is the consistency and dissimilarity between multimodal data, which leads to unequal detection contributions. To address this dilemma, we propose the heterogeneous adaptive fusion (HAF) module to fuse multimodal decision-making jointly. In this study, we proposed a heterogeneous fusion network for MCD (HF-MCD) with the HCF and the HAF module. We validate the proposed method on four publicly available MCD datasets. Extensive experimental results have demonstrated the superior performance of HF-MCD over the state-of-the-art methods.