Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement

Remote sensing image segmentation is a fundamental task in Earth observation. Rapid development has been made in the past decade owing to the deep learning techniques. Most of the existing methods assume that the training and inference remote sensing images hold the identical and independent distribution. However, in practical cases, the pretrained segmentation model has to deal with remote sensing images that have not seen before. In this article, we advance this bottleneck by introducing the domain generalization (DG) paradigm to remote sensing image segmentation. It does not need to involve any images from the target domain during training, compared with the existing domain adaptation (DA) paradigm. This article focuses on the key challenges from the various instance size and dramatic image appearance from cross-domain remote sensing images, and proposes a novel multiscale instance disentanglement learning scheme. Technically, it consists of three key components, namely, multiscale instance encoding (MSIE), domain-invariant instance disentanglement (DII), and domain generalized semantic decoding (SD). The proposed MSIE module incorporates both depth convolution and multiscale representing, so as to learn robust semantic representation despite the cross-domain scale variation. The DII module, on the other hand, helps eliminate the impact of cross-domain styles. Finally, the domain generalized SD fuses the style-invariant representation and outputs the per-pixel semantic prediction. Extensive experiments on four cross-domain settings show its effectiveness over existing DA and DG methods from both remote sensing and computer vision community, yielding an mean intersection over union improvement of 3.3% and 6.0% on the Postdam and Vaihigen unseen target domain over the second best, respectively.