Unleashing Fourier-Domain Potential: Spatial–Spectral Reconstruction Framework for Remote Sensing Pansharpening

Pansharpening aims to generate high-resolution multispectral (HR-MS) images by fusing the corresponding low-resolution multispectral (LR-MS) and high-resolution panchromatic (PAN) images. While the spatial–spectral properties modeling plays a crucial role in generating high-quality HR-MS images, existing approaches suffer from: 1) directly modeling the entangled spatial–spectral properties and 2) lacking task-specific priors for spatial and spectral properties modeling. This article proposes a novel Fourier domain-based approach (Fourier-SSR) to address these problems, where phase and amplitude components of PAN and LR-MS are considered to individually model spatial and spectral properties for the target HR-MS image. Our Fourier-SSR is motivated by the crucial findings in Fourier domain: 1) manifestation of spatial–spectral properties, that is, the spatial and spectral properties of remote sensing images can be individually manifested in their phase and amplitude components in Fourier domain; 2) spatial property-related prior, that is, only reconstructing the phase component of PAN image in Fourier domain, could generate spatial property required for target HR-MS images; and 3) spectral property-related prior, that is, jointly models the amplitude components of PAN and LR-MS images could generate the required spectral property for target HR-MS images. Based on the aforementioned findings, we design a Fourier-guided spatial mixer (FSpa-Mixer) and a Fourier-guided spectral mixer (FSpe-Mixer), which innovatively employ complex feature interaction strategies to individually reconstruct the phase and amplitude components for target HR-MS. Experiments show that our methods unleash Fourier domain potential in individually modeling spatial and spectral properties for the target HR-MS image, leading to superior performance over previous state-of-the-art (SOTA). Our code is provided in the https://github.com/Florina2333.