SemiBaCon: Semi-Supervised Balanced Contrastive Learning for Multimodal Remote Sensing Image Classification

The limited availability of annotated training data significantly constrains the classification accuracy of hyperspectral image (HSI) and LiDAR fusion approaches. Although contrastive learning (CL) has emerged as a potential solution, current implementations frequently neglect the critical class imbalance issues during unlabeled sample selection. To address the issue, we introduce a novel semi-supervised balanced contrastive learning (SemiBaCon) framework for multimodal remote sensing (RS) image classification. First, we propose a superpixel-based balanced sampling (SPBS) mechanism that fundamentally addresses class imbalance through intelligent pseudo-label (PS) generation. By segmenting the HSI data into homogeneous superpixels and implementing intraregion label propagation, the method ensures statistically balanced PS selection across categories, effectively overcoming the bias introduced by conventional random sampling strategies. Second, our architecture integrates a dual-stream encoder combining convolutional neural networks (CNNs) with Transformers, enabling hierarchical feature extraction from spectral–spatial characteristics of HSIs and elevation patterns of LiDAR. This design facilitates the construction of multimodal positive sample pairs, achieving enhanced representation learning through intermodal consistency constraints. Third, we develop a pseudo-label guided CL (PLCL) paradigm that synergistically combines PS confidence with feature similarity metrics, which effectively reduces intraclass variance and improves decision boundaries in the latent space. Comprehensive evaluations on three benchmark datasets demonstrate the framework’s superior performance compared to the state-of-the-art methods.