Two-Stream Networks for Contrastive Learning in Hyperspectral Image Classification

In the domain of hyperspectral image (HSI) classification, the majority of deep learning methods have necessitated a substantial number of manually annotated samples to achieve outstanding results. However, the process of annotating HSI is conducted at the pixel-level, rendering it not only time-consuming but also financially burdensome. In light of this circumstance, contrastive learning methods that harness unlabeled samples by assigning pseudolabels through pretext tasks have garnered significant attention. Nevertheless, current contrastive learning methods primarily concentrate on exploring spatial diversity among surface samples in natural images, while neglecting the spectral diversity of point targets in HSI, resulting in insufficiently comprehensive feature exploration. In addition, due to the distinct learning objectives between upstream and downstream tasks, this leads to insufficient generalization when transferring to downstream tasks. To tackle these challenges, we propose a two-stream contrastive learning network for few-shot HSI classification. During the pretraining phase, one stream is deployed to probe spatial diversity among samples, whereas the other stream delves into spectral diversity. Subsequently, for transferring to downstream classification tasks, a multilevel fusion network was introduced. It can integrate shallow network features with higher generalization capabilities and deeper network features that are more task-specific. The fused features exhibit an improved performance when employed for classification tasks. Experimental results on four publicly available datasets illustrate that our approach outperforms state-of-the-art methodologies.