Hybrid Spectral-Spatial Convolutional Network and Transformer with Mixup Regularization for Hyperspectral Image Classification

Existing deep learning-based models for hyperspectral image classification (HSIC) may be suboptimal in the utilization and the balance between the spatial and spectral information, and they pay more attention to the design of architecture and modules but ignore the generalization of the model on the data with slight distribution shift. In this paper, a novel framework based on spectral-spatial convolutional Transformer and Mixup regularization (SSCT-M) is proposed for hyperspectral image classification. SSCT-M designs a dual-branch architecture to extract spectral and spatial features and adopts a Transformer-based network to adaptively balance the contribution between spectral and spatial pipelines. Then, SSCT-M adopts a two-stage mixup regularization to supervise the learning of the model, which can boost the generalization performance of the model by learning convex combinations on the labeled data while encouraging the model to make consistent predictions on the perturbed unsupervised data. Extensive experimental results on two HSI benchmarks demonstrate the effectiveness of our proposed framework.