High-Fidelity Pansharpening via Trigeminal Pyramid Decoding of CNN-Transformer Encoded Features

Spectral and spatial fidelity remains a longstanding challenge in the field of pansharpening, which aims to generate high-resolution multispectral (HRMS) images by integrating high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images. This study proposes a high-fidelity pansharpening network that utilizes bidirectional trigeminal pyramid decoding of features encoded by a CNN-Transformer architecture. Specifically, local and global features at multiple scales are initially extracted using a convolutional neural network (CNN)–Transformer encoder to facilitate multiscale feature fusion. Subsequently, we design a decoder based on bidirectional trigeminal pyramids to achieve a high-fidelity fusion output. One reverse decoding pyramid decodes the fused features of LRMS and PAN images from the encoder. One spectral feature pyramid is employed to enhance the spectral information of the reverse decoding pyramid, while the last spatial feature pyramid is utilized to enrich the spatial information, thereby improving the overall spectral and spatial fidelity of the fused output. Furthermore, content-guided attention (CGA) is incorporated to adaptively integrate the spectral and spatial feature pyramids into the reverse decoding pyramid. Extensive experiments demonstrate that our network surpasses the comparative state-of-the-art (SOTA) methods in both qualitative and quantitative evaluations. The code is available at https://github.com/songvvvv/pansharpening