Exploring Hybrid Contrastive Learning and Scene-to-Label Information for Multilabel Remote Sensing Image Classification

Multilabel remote sensing (RS) image classification aims to predict multiple semantic labels from an RS image. Previous methods [e.g., graph convolution networks (GCNs)] focus on mining the relationships of multiple labels, neglecting that the scene information is closely related to labels. To remedy this deficiency, in this article we propose a novel end-to-end deep neural network for multilabel RS image classification. In the proposed network, we use the GCN as the base model and introduce several new components to improve the classification performance. First, we explore hybrid contrastive learning (CL), including supervised transformation-based CL and unsupervised mix-based CL, to explicitly learn discriminative scene representations. Then, we apply the GCN-based classifier to the learned scene representations to obtain initial label prediction scores. Meanwhile, we pass the scene representations to a softmax layer to predict the probability that each image belongs to each specific scene class and use the scene-to-label information with the law of total probability to calibrate the initial label prediction scores. Finally, we incorporate CL, scene classification, and multilabel classification into a unified learning framework using uncertainty to weigh different losses. Experimental results on two benchmark RS datasets demonstrate the superiority of our proposed network for multilabel image classification.