Cross-Modality Domain Adaptation Based on Semantic Graph Learning: From Optical to SAR Images

Synthetic aperture radar (SAR) imaging provides a distinct advantage in scene understanding due to its capability for all-weather data acquisition. However, in comparison to easily annotated optical remote sensing images (RSIs), the lower imaging quality of SAR images presents significant challenges in obtaining manually annotated training data, which poses substantial issues for SAR image analysis. In this article, we employ the domain adaptation (DA) that leverages labeled optical images to better understand unlabeled SAR images. Global feature alignment as a method for DA has demonstrated effectiveness in transferring knowledge, yet it faces challenges in cross-modality adaptation from optical remote sensing to SAR images due to their differing imaging mechanisms. With distinct visual features between optical and SAR images, the semantic dependence is difficult to construct, which results in low-quality pseudolabel assignment for SAR images. To address the above issue, we propose a semantic graph learning framework (SGLF) to comprehensively align the global features of optical remote sensing and SAR images by modeling the cross-modality semantics and generating high-quality pseudolabels. It can be applied for SAR scene classification and object detection when only optical RSIs are labeled. Specifically, a cross-modality semantic graph alignment (CSGA) module is constructed to model and align the second-order semantic dependencies by aggregating cross-modality visual semantic information. Then, an uncertainty-based robust pseudolabel generation (URPG) module is designed to generate pseudolabels for effective semantic alignment and self-training by modeling the uncertainty of pseudolabels for each SAR image. Comprehensive experiments show that our proposed method outperforms the state-of-the-art methods on scene classification (NWPU-RESISC $45~\rightarrow $ WHU-SAR6, MLRSNet $\rightarrow $ NWPU-SAR6, MLRSNet $\rightarrow $ NWPU-SAR6, and NWPU-RESISC $45~\rightarrow $ NWPU-SAR6) and object detection (MASATI-ship $\rightarrow $ SSDD, MVSRD $\rightarrow $ SARDet-vehicle, and DIOR-airplane $\rightarrow $ SAR-airplane) tasks. The code and datasets are publicly accessible at https://github.com/XZhang878/SGLF