SpectralDINO: Dual Mixture-of-Subspaces Low-Rank Adaptation for Cross-Domain Hyperspectral Image Few-Shot Classification

Recently, few-shot learning (FSL)-based methods have achieved impressive results in cross-domain hyperspectral classification. However, existing approaches often ignore differences in spectral information caused by varying spectral range across different datasets and encounter limitations due to the constrained parameter size of the models. Furthermore, the substantial differences between RGB and hyperspectral images (HSIs) present significant challenges in applying foundation models (e.g., SAM and DINOv2) to the hyperspectral domain. This article proposes a novel framework for cross-domain few-shot hyperspectral classification that leverages parameter-efficient fine-tuning (PEFT), which we apply to DINOv2 to construct SpectralDINO. Different spectral ranges reflect different physical properties of the target. To enhance the consistency of the spectral features extracted by the model from different domains, we introduce a source-domain spectral alignment (SDSA) strategy to align the spectral ranges and bands of the source-domain data to the target domain. SpectralDINO employs the visual foundation model to enhance its ability to generalize cross-domain knowledge. Additionally, we propose a dual mixture-of-subspaces low-rank adaptation (Dual-MoS LoRA) method to address the structural limitation of the LoRA methods in distinguishing domain-specific features from multidomain inputs. Only 1.14% of the 21.37 M parameters need to be trained to perform fine-tuning. Extensive experimental results on three public datasets demonstrate the superiority of SpectralDINO.