Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

The weakly supervised semantic segmentation (WSSS) method aims to assign semantic labels to each image pixel from weak (image-level) instead of strong (pixel-level) labels, which can greatly reduce human labor costs. However, there are some problems in WSSS of remote sensing images, such as how to locate labels accurately and how to get precise segmentation edges. To address these issues, we propose a novel framework directly transferring the scene classification model to perform semantic segmentation. We first train a multilabel scene classification network as the encoder to obtain the pretrained model, and then, the feature learned by the model is transferred to the decoder. Different from other methods, we propose a saliency map generator (SMG) instead of the class activation map (CAM) for more accurate location information by making pixels belonging to the same class lie close together while different classes are separated in feature space. Meanwhile, we take the superpixel patch as processing unit to provide precise boundary inhibition for the saliency map. To assign semantic labels for each patch, combined with extracted salient region, we propose a module responsible for exploiting the consistency of spatial and semantic similarity between different patches. Finally, we incorporate the above two modules to supervise the training process of the decoder without generating pseudolabels as most methods do, thus simplifying the training process. Experimental results show that our method outperforms other weakly supervised approaches on dense labeling remote sensing dataset (DLRSD) and Wuhan dense labeling dataset (WHDLD) with at least a 3% improvement on mean intersection over union (mIoU).