Unsupervised Model-Embedded Two-Stage Diffusion Method for Multispectral and Hyperspectral Image Fusion

The multispectral and hyperspectral image fusion tasks aim to obtain the hyperspectral image(HSI) with a high spatial resolution. However, existing fusion methods usually utilize degradation simulation for training due to the inaccessible ground truth, which inevitably leads to spatial distortion and parameter bias during application. Besides, neglect of the prior information inherent in inputs and the deep-learning network’s low-frequency preference eventually leads to lower interpretability and limited performance. To this end, we proposed a model-embedded two-stage diffusion method(MTDiff) for unsupervised reconstruction of high spatial resolution HSI. In the first stage, the degradation model is estimated by the prior information inherent input pairs. Meanwhile, in the second stage, embedded with the degradation model, the dual-resolution diffusion model reconstructs high-resolution HSI. Specifically, treating the degradation process as a fixed diffusion step, an unsupervised paradigm is established through a mapping from upsampled low-resolution HSI to high-resolution HSI. Besides, with the estimated degradation model, the well-designed dual-resolution diffusion model step-by-step perturbs and then denoises the image in both native and degraded resolutions for a fidelity reconstruction with great interpretability. Furthermore, to establish a high-frequency shortcut for network learning, a discrete cosine injection module is designed to flatten the frequency information to a clear 2D domain with a huge high-frequency area, achieving sharp textures and clear structures in fusion results. Extensive systematic experiments across three datasets indicate the superior performance of MTDiff in multispectral and hyperspectral fusion tasks.