CNN and Transformer Hybrid Network for Hyperspectral Image Classification

Hyperspectral images (HSIs) contain a wealth of information and have important applications in the fields of military, agriculture, medicine and et al. The excellent local feature representation ability of convolutional neural network (CNN) makes it achieve good classification results in hyperspectral image classification tasks, but its ability to capture global features is limited and the computational cost is high. The Transformer’s ability to represent global features can largely make up for the above shortcomings of CNNs. Therefore, in this paper, we propose a hybrid network based on CNN and transformer (CTHN) for HSI classification. Firstly, the multi-scale spectral and spatial features were preliminarily extracted through the multi-scale spectral-spatial feature extraction module. Then, the deformable convolutional residual module is introduced to enhance the adaptability of the network to various complex shapes of ground objects, and further learn complex spatial information. Finally, the transformer module is used to deeply represent and learn global features. A series of experimental results on three publicly available datasets show that the classification performance of CTHN outperforms several state-of-the-art comparison algorithms.