Hybrid Transformer Architecture for Spectral Super-Resolution Reconstruction of Multispectral Images

Spectral super-resolution technology, which reconstructs 31-band hyper-spectral images from RGB natural scene images within the 400-700nm bands, has seen rapid growth. However, its fixed spectral resolution and spectral coverage limit its application in remote sensing imaging, particularly for aerial images with multi-band information. The lack of corresponding high-spectral image pairs has hindered research progress, leaving the potential spectral information of these remote sensing images untapped. In this study, we explore a hybrid transformer architecture for multispectral images that carry visible light and near-infrared informations to achieve spectral super-resolution. This network integrates both intra-row and intra-column attention mechanisms, along with a cross inter-row and inter-column attention mechanism, to precisely capture and process the spatial and spectral features in spectral images. In the case of two simulated datasets, the experimental results demonstrate favorable outcomes. In classification experiments using multimodal Pavia University datasets, the reconstructed hyper-spectral images exhibit superior performance with higher average accuracy (95.30%), overall accuracy (95.70%), and Kappa coefficient (93.50%).