MSSCFormer: Multigranularity Spatial–Spectral Convolution Transformer Network for Hyperspectral Image Classification

Recently, convolutional neural networks (CNNs) and Transformers have achieved considerable success in hyperspectral image (HSI) classification tasks. However, existing methods not only lack the study of spectral variability of samples from the same land class but also struggle to mine local–global spectral information and spatial structure information of HSI at different granularities effectively. To mitigate these limitations, this article proposes a multigranularity spatial–spectral convolution Transformer (MSSCFormer) network, which can reduce the intra-class spectral differences of samples and extract multigranularity spatial–spectral features from a local-global-local perspective. Specifically, MSSCFormer consists of three components: intra-class spectral attention (ICSA), spatial–spectral feature extractor (SSFE), and global–local convolutional Transformer (GLCT). First, ICSA redistributes the spectral weights of samples of the same land class by establishing an attention mapping between spectral channels within the class to reduce the intra-class spectral differences. Second, SSFE extracts shallow spatial features and multigranularity spectral features of HSI with reassigned spectral weights samples from a local perspective. Finally, GLCT takes advantage of CNNs and Transformers; it uses MHSA to model global spectral features and utilizes local context feature block (LCFB) to capture local spatial features in a multigranularity way. Experimental results on three benchmark datasets show that MSSCFormer exhibits excellent performance on the HSI datasets and outperforms state-of-the-art HSI classification algorithms.