Key Feature Repairing Based on Self-Supervised for Remote Sensing Semantic Segmentation

As one of the fundamental issues in remote sensing, semantic segmentation has always received widespread attention. However, different from natural images, remote sensing images contain more complex category information, which poses many challenges to researchers, e.g., the lack of large-scale labeled semantic segmentation datasets on remote sensing and the accurate distinguishment of the edge areas between different classes. Recently, self-supervised methods have tried to avoid the issue of great dependency on labeled datasets. However, existing self-supervised methods were typically based on randomly masking and repairing images to learn features from unlabeled images. Random masks cannot drive the model focus on the salient information of the image, thus the learned features are not representative. In this letter, to improve the accuracy and generalization ability of the model in remote sensing semantic segmentation, we propose a key feature repairing network based on self-supervised learning (SSL), called KFRNet. KFRNet calculates the similarity between each image patch and its surrounding patches and sorts them to find the patches with more prominent feature information for masking and repairing, effective obtaining image context information. Besides, to improve the model’s ability to distinguish different classes of objects, we designed an image comparison branch to obtain the category features of the image by comparing positive and negative samples. The experimental results on the POTSDAM and LoveDA datasets show that the proposed method can effectively improve segmentation accuracy. The overall accuracy (OA), mean intersection over union (MIOU), and Fscore indices reached 89.73%, 83.96%, 91.15% (POTSDAM) and 70.81%, 53.40%, 68.86% (LoveDA), even surpassing some supervised learning methods.