FSGformer: Frequency Separation and Guidance Transformer for Pansharpening

Pansharpening is a crucial task in remote sensing image processing, aiming to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) images with high-resolution panchromatic (PAN) images. However, most current deep learning methods for pansharpening rarely consider the frequency differences in PAN and MS images effectively, resulting in harmful mixing of frequency information and inefficient learning of features. Furthermore, frequency separation-based methods continue to face challenges such as insufficient consideration of the relationship between frequency and spatial information, amplification of noise due to separation, and inadequate learning of frequency information. To address these problems, we propose a novel frequency separation and guidance Transformer, named FSGformer, which focuses on the differences and interactions between high- and low-frequency components. Specifically, we design an adaptive frequency separator tailored for pansharpening to effectively differentiate between distinct frequencies. Subsequently, we develop a carefully designed guidance module that enables the fusion process to benefit from the interaction of frequency information. In addition, we introduce a novel Transformer module that features a joint spatial and spectral attention mechanism and integrate it into a meticulously crafted network architecture to support the effective representation of different frequency information, thereby generating high-quality fused results. Moreover, we incorporate a hybrid frequency separation (HFS) loss to enhance overall performance. Extensive experimental evaluations have confirmed the superiority and generality of our FSGformer. The code is available at https://github.com/lqrscode/FSGformer.