Enhancing the Semi-Supervised Semantic Segmentation With Prototype-Based Supervision for Remote Sensing Images

While image semantic segmentation is a fundamental and well-studied task in remote sensing (RS) society, it usually depends on large amounts of pixel-level annotations. RS image semi-supervised semantic segmentation (RSIS4) tries to improve performance by exploring the unlabeled data, thus significantly reducing the label costs. The core idea of RSIS4 is to transfer the prior information from the labeled to unlabeled pixels, which is commonly achieved by considering the confident part of the softmax prediction as pseudolabels for further supervised learning. However, such pixel-level instruction could inevitably involve uncertainty (e.g., noise and error) due to the extremely limited annotated data at the initialization. To address this issue, in this letter, we employ the prototypes, which contain inbuilt resistance to potentially inaccurate pixels, to bring substantial supervision directly from the embedded feature space. Specifically, we project deep features into the embedding space to generate prototypes, each of which can be regarded as the category-level feature representation of a certain semantic category. These prototypes are then used to perform the pixelwise classification, with the advantage of capturing the global similarity throughout the whole pixels within the category. Moreover, to ensure accurate prototypes, we further introduce pixel-prototype contrast to better explore the discriminative category-level feature embedding. By integrating the guidance from the above pixel-level and category-level feature representations, the proposed algorithm obtains high-quality pseudolabels and extracts effective features. Extensive experiments on four RS image segmentation datasets have demonstrated the effectiveness of the proposed method. The code is available at https://github.com/Duckyee728/PCSSS.git.