Unsupervised Across Domain Consistency- Difference Network for Hyperspectral Image Super-Resolution

Without reducing the spectral resolution, hyperspectral image (HSI) super-resolution has achieved remarkable progress thanks to the success of deep neural networks. However, existing methods cannot fully excavate the latent high-frequency details only in the single spatial domain. Different from existing methods that only achieves the super-resolution task in spatial domain, we optimize the amplitude spectrum and phase spectrum in frequency domain to obtain high-resolution HSI (HR-HSI). We propose a new unsupervised framework to reconstruct HR-HSI using only the observed low resolution (LR) HSI and HR multispectral image (MSI). Based on triple-level modeling, the encoder–decoder learns abundant features including contextual information from multiple scales. In addition, we propose iterative across domain consistency-difference (ADCD) module, which is embedded between encoder and decoder. In ADCD module, three parallel convolution streams (amplitude spectrum adjustment branch, phase spectrum adjustment branch and spatial domain branch) are used to explore the consistency-difference between each other, which is preserved by memory units within the module. Particularly, we embed the dilated causal convolution in the frequency domain processing branch, which is convenient to flexibly adjust the receptive field and adapt to different domains. Extensive experiments are conducted on widely used datasets in comparison with state-of-the-art models, demonstrating the advantage of the proposed method.