Dual-Phase Framework for Few-Shot Hyperspectral Image Classification With Spatiospectral Masked Autoencoder and Episode Training

This article introduces a two-phase learning approach for hyperspectral image (HSI) classification using few-shot learning (FSL). For the first phase, we present a novel spatiospectral masked autoencoder (ssMAE)—an advanced self-supervised learner. For the ssMAE backbone network, we designed a transformer encoder-decoder network, where we replaced the linear layer that is used as the initial feature embedding with a 3-D convolutional layer to better extract local spectral-spatial features from 3-D visible sub-patches. By tapping into vast unlabeled data, the ssMAE learns general HSI features. In the second phase, the ssMAE encoder is fine-tuned to extract discriminative features for classification using the few-shot labeled training samples. This is achieved through a unique hybrid episode learning method that integrates the ssMAE encoder in a prototypical network (PN). We innovate with a mix of global and local prototypes (combined global-local (CGL) prototype) to refine label predictions. This technique maximizes data usage, focuses on specific samples, and mitigates issues from subpar episodes. Tested on three HSI datasets, our approach outperforms alternative few-shot methods. The code will be made publicly available at https://github.com/Weejaa04/SSMAE.