Semi-Supervised Learning for Joint SAR and Multispectral Land Cover Classification

Semisupervised learning techniques are gaining popularity due to their capability of building models that are effective, even when scarce amounts of labeled data are available. In this letter, we present a framework and specific tasks for self-supervised pretraining of multichannel models, such as the fusion of multispectral and synthetic aperture radar (SAR) images. We show that the proposed self-supervised approach is highly effective at learning features that correlate with the labels for land cover classification. This is enabled by an explicit design of pretraining tasks which promotes bridging the gaps between sensing modalities and exploiting the spectral characteristics of the input. In a semisupervised setting, when limited labels are available, using the proposed self-supervised pretraining, followed by supervised fine-tuning for land cover classification with SAR and multispectral data, outperforms conventional approaches such as purely supervised learning, initialization from training on ImageNet, and other recent self-supervised approaches.