Exploring Text-Guided Information Fusion Through Chain-of-Reasoning for Pansharpening

Pansharpening aims to enhance the spatial resolution of low-resolution multispectral (LRMS) images by integrating high-frequency information from a corresponding texture-rich panchromatic (PAN) image, while maintaining the spectral integrity of the LRMS image. Although text-guided multimodal learning has made considerable strides in the natural image domain, its potential for pansharpening remains underexplored, primarily due to the limited availability of multimodal remote sensing datasets. To this end, we construct an entirely new pansharpening framework by making efforts from three key aspects: 1) text-equipped multimodal data collection through chain-of-reasoning; 2) large model prior-driven multimodal information fusion; and 3) visual information interaction through prompt engineering, leveraging textual information to guide the pansharpening process within a multimodal fusion framework. We initially utilize the generic large language model priors to generate descriptive captions for multispectral (MS) images, forming a multimodal pansharpening dataset. By integrating super-resolved imagery and segmentation maps generated by segment anything, we apply chain-of-thought (CoT) prompting to generate spatially focused captions across diverse satellite datasets. These captions enhance visual features and provide high-level contextual information, improving semantic understanding for pansharpening. Building on the aforementioned multimodal data, we tailor two text-guided information fusion modules: a textual enhancement block (TEB) standing on a large model prior and a textual modulated block (TMB) utilizing text information to effectively guide and refine the pansharpening fusion process. Extensive experiments on multiple satellite datasets demonstrate that our proposed framework outperforms state-of-the-art (SOTA) methods, highlighting its effectiveness and superior performance in pansharpening.