Sample-Level Improved Cross-Source Contrastive Learning for PAN and MS Joint Classification

In recent years, the number and ways of acquiring panchromatic (PAN) images and multispectral (MS) images have increased, and manual labeling costs have also increased. Processing these data efficiently has become a challenge. In this article, we propose a sample-level improved cross-source contrastive learning (CL) method for PAN and MS joint classification (SLCL), which aims to provide a self-supervised pretraining model using unlabeled samples for downstream joint classification using a small number of labeled samples. First, we propose a sample weighting and screening (SWS) strategy, which enables the model to learn inter- and intrasource sample representations, while balancing the interference from false samples so that the model learns true samples. It solves the problems of homologous similar features embedded far away and false-negative samples bringing the wrong learning direction, which exist in existing CL methods. In addition, we design a hard sample learning (HSL) module for the problem of mining and optimization of hard samples. The module efficiently mines hard samples and uses a new loss function to make the model more focused on hard sample optimization. It further improves the accuracy of pretraining models for downstream tasks. Our method performs best on multiple datasets, and it is experimentally validated and analyzed. The code is available at https://github.com/Xidian-AIGroup190726/SLCL