Few-Shot Open-Set Hyperspectral Image Classification With Adaptive Threshold Using Self-Supervised Multitask Learning

Existing hyperspectral image (HSI) classification methods rarely consider open-set classification (OSC). Although some reconstruction-based methods can deal with OSC, they lack adaptive threshold strategies and heavily rely on the labeled samples. Therefore, this article proposes a self-supervised multitask learning (SSMTL) framework for few-shot open-set HSI classification, including three stages: pretraining stage (PTS), fine-tuning stage, and testing stage. The model consists of three modules: data diversification module (DDM), 3-D multiscale attention module (3D-MAM), and adaptive threshold module (ATM), as well as a backbone network: dense feature pyramid network (DFPN). In the PTS, we construct a self-supervised reconstruction task with unlabeled samples for model initialization, where DDM aims to improve the robustness of the model and 3D-MAM applies 3-D multiscale convolution to focus on key information spatially and spectrally. In the fine-tuning stage, we further optimize the model with a few labeled samples based on both reconstruction task and classification task, where ATM implements adaptive threshold strategies based on uncertainties of predicted probability and reconstruction loss, and DFPN is helpful to retain the detailed information. The experimental results on three common HSI datasets show SSMTL performs significantly well and even surpasses many advanced closed-set and open-set HSI classification methods.