I2MEP-Net: Inter- and Intra-Modality Enhancing Prototypical Network for Few-Shot Hyperspectral Image Classification

Prototypical network, celebrated for its flexible network structure and metric computation capability, has become a prevailing strategy in addressing challenges associated with few-shot hyperspectral image classification (HSIC). However, factors, such as insufficient training samples, spectral mixing, and noise interference, in complex scenarios severely impact the stability of its prototypes, ultimately leading to a degradation in classification performance. Therefore, this article proposes a novel method called inter- and intra-modality enhancing prototypical network (I2MEP-Net), which incorporates two auxiliary modalities to facilitate both inter- and intra-modality enhancing prototype learning (Intra-MEPL) with the base modality. Specifically, I2MEP-Net employs the auxiliary LiDAR modality with base HS modality from the same scene for inter-modality enhancing prototype learning (Inter-MEPL), which offsets the sparsity of few-shot features through a cross-modal approach. In addition, it utilizes the target few-shot labeled data as the auxiliary HS modality for Intra-MEPL on the enhanced prototypes, in a way that adaptively generates diversity features, thereby further enriching the prototype embedding space and achieving more fine-grained and stable prototypes. Comprehensive experiments are conducted on the publicly available hyperspectral (HS) image datasets. These experiments indicate that the proposed I2MEP-Net outshines the existing state-of-the-art deep learning techniques and few-shot classification methodologies.