S3A: A Self-Supervised Saliency Analysis Framework for Hyperspectral Image

Most existing visual saliency analysis methods are meticulously crafted for natural RGB images, which typically lean upon foundational spatial visual cues, such as contrast, structure, and texture in scenes. However, these methods exhibit limitations in spectral saliency analysis due to the existence of numerous materials that may manifest identical RGB values while harboring disparate spectral characteristics. Consequently, it is an exigency to introduce the saliency method, especially for hyperspectral images (HSIs). In this article, we establish a set of hyperspectral saliency principles that incorporate both spectral and spatial attributes, and accordingly present a novel self-supervised saliency analysis (S3A) framework for HSIs. Note that our S3A is designed as a discriminant architecture composed of three interconnected components. More specifically, an unsupervised HSI representative patch sampling (RPS) module is designed to pick up some representative pixels for S3A training. Subsequently, we construct a classification-based patch spectral discrimination network (CPSD-Net) to evaluate the HSI saliency. Finally, a patch-to-global spatial diffusion (P2G-SD) module is constructed to diffuse the saliency from few supervised samples to the other HSI pixels. Moreover, to demonstrate the performance of our saliency analysis framework, we apply the HSI saliency to some classic downstream tasks including band selection (BS) and HSI classification. On several popular HSI datasets, the satisfactory quantization results fully verify the rationality and effectiveness of our S3A framework, in terms of entropy value and mean spectral divergence (MSD) of the selected bands in BS task, as well as the accuracy in the HSI classification task. The source code is publicly available at https://github.com/Lee-zl/S3A