Unsupervised Domain Adaptation Semantic Segmentation of Remote Sensing Images With Mask Enhancement and Balanced Sampling

Unsupervised domain adaptation (UDA) aims to improve model performance in the target domain by leveraging labeled data from the source domain while not requiring labeled data in the target domain. It has been widely applied in cross-domain semantic segmentation of remote sensing images (RSIs). Despite some advancements in this area, challenges such as class confusion due to color and texture similarities, class imbalance due to significant scale variations and sample imbalance continue to impede progress in UDA for RSI segmentation. To address these challenges, we propose a novel self-supervised teacher-student network framework, including two innovative techniques: mask-enhanced class mix (MECM) and scale-based rare class sampling (SRCS). The MECM method applies a high proportion of masks to mixed images derived from both source-domain images and target-domain images, which encourages the model to infer the semantic information of masked areas from the surrounding context, enhancing cross-domain contextual semantic learning and improving the recognition accuracy of similar classes. Additionally, SRCS increases the sampling proportion of small-scale rare classes, mitigating the issue of class imbalance. Experiments show that our method outperforms existing UDA techniques in terms of PA, mF1, and mIoU, achieving state-of-the-art results on three public datasets. Notably, in the Potsdam IRRG to Vaihingen UDA scenario, our method’s performance on the key metric, mIoU, even surpasses that of supervised training, demonstrating the superiority of our approach. Codes are available at https://github.com/Qiuyb-ai/UDA-With-ME-and-BS.

I. Introduction

With the number of remote sensing images (RSIs) increasing extensively and rapidly, deep learning semantic segmentation technologies have undergone significant advancement, especially in remote sensing surface monitoring [2]. Semantic segmentation has been a crucial technique in satellite image processing [3], aiming at pixel-level classification of various semantic targets in an image. It has proved to be highly effective in applications such as land-cover classification [4], [5], road extraction [6], building detection [7], and farmland segmentation [8] using RSIs.