CR-SSRNet: Cross-Sensor Robust Spectral Super-Resolution Network Guided by Cognition Features

Spectral Super-Resolution (SSR) aims at reconstructing a latent hyperspectral images (HSI) from a RGB image. Recent progress mainly focused on building a deep spectral super-resolution networkto directly map the input RGB image to the corresponding HSI. Their pleasing performance depends on the assumption that the spectral response function determined by the RGB sensor is consistent across training and test data. However, in practice, the training and test data are inevitably captured by different RGB sensors, thus resulting in obvious performance drop when using these networks. To mitigate this problem, we present a novel cognitive feature guided cross-sensor robust spectral super-resolution network. In a specific, a U-shape multi-scale network is first established to learn the deep mapping between input RGB image and the latent HSI. Then, a large-scale foundation cognitive model is introduced to extract multi-level cross-sensor invariant cognitive features from the input RGB. Moreover, these features are separately adapted and injected into different decoder blocks in the U-shape spectral super-resolution network. By doing these, the proposed network learns to appropriately guide the coarse-to-fine spectral reconstruction process using multilevel cognitive features, and thus shows better generalization performance in the cross-sensor SSR tasks. Experiments on two benchmark datasets demonstrate the superiority of the proposed method over several state-of-the-art baselines.