SeG-SR: Integrating Semantic Knowledge Into Remote Sensing Image Super-Resolution via Vision-Language Model

High-resolution (HR) remote sensing imagery plays a vital role in a wide range of applications, including urban planning and environmental monitoring. However, due to limitations in sensors and data transmission links, the images acquired in practice often suffer from resolution degradation. Remote sensing image super-resolution (RSISR) aims to reconstruct HR images from low-resolution (LR) inputs, providing a cost-effective and efficient alternative to direct HR image acquisition. Existing RSISR methods primarily focus on low-level characteristics in pixel space, while neglecting the high-level understanding of remote sensing scenes. This may lead to semantically inconsistent artifacts in the reconstructed results. Motivated by this observation, our work aims to explore the role of high-level semantic knowledge in improving RSISR performance. We propose a semantic-guided super-resolution (SR) framework, semantic-guided super-resolution (SeG-SR), which leverages vision-language models (VLMs) to extract semantic knowledge from input images and uses it to guide the SR process. Specifically, we first design a semantic feature extraction module (SFEM) that utilizes a pretrained VLM to extract semantic knowledge from remote sensing images. Next, we propose a semantic localization module (SLM), which derives a series of semantic guidance from the extracted semantic knowledge. Finally, we develop a learnable modulation module (LMM) that uses semantic guidance to modulate the features extracted by the SR network, effectively incorporating high-level scene understanding into the SR pipeline. We validate the effectiveness and generalizability of SeG-SR through extensive experiments: SeG-SR achieves state-of-the-art performance on three datasets, and consistently improves performance across various SR architectures. Notably, for the $\times 4$ SR task on the UCMerced dataset, it attained a PSNR of 29.3042 dB and an SSIM of 0.7961. Codes can be found at https://github.com/Mr-Bamboo/SeG-SR