Robust Hyperspectral Image Classification Using a Multiscale Transformer With Long- and Short-Distance Spatial–Spectral Cross Attention

Hyperspectral images (HISs) have continuous spectra that can be used to accurately identify the land cover contained within them. Generally speaking, small-scale spectral features can reveal local information within the spectral sequence, while large-scale ones capture global information across spectra. Recently, the proposed Transformer-based hyperspectral image classification (HIC) methods have shown superiority over convolutional neural networks (CNNs) in processing long spectral sequences but struggle with extracting local spectral details. Meanwhile, the multiscale information of the spectral dimension has not been fully explored. To address these issues, we propose a multiscale spatial–spectral Transformer named LSDnet. First, a bilateral filtering-based feature enhancement (BFFE) module is utilized to promote the reliability of shallow features. Then, a long- and short-distance spatial–spectral crossattention (LSDSC) module is proposed to capture both local and global HSI features. Moreover, a multiscale spectral embedding (MSSE) module is introduced to enrich local details among adjacent spectral bands. To enhance the model generalization, the spectral position bias (SPB) coding is designed in the Transformer encoder, adapting to variable spectral numbers of different HSIs. Furthermore, the traditional data partitioning strategy for HIC suffers from information leakage. Therefore, we propose a new data partitioning method to prevent the overlapping between the training and testing data. Meanwhile, an adjustment factor is utilized to balance the number of samples in each category, while a dynamic equilibrium loss (DEL) function is proposed to ensure the training contribution of every category. Experiments on three public datasets validate the rationality and effectiveness of our proposed HIC method. The codes will be available at: https://gitee.com/pdypdy/lsdsc_-tgrs.git.