Satellite Image Synthesis From Street View With Fine-Grained Spatial Textual Guidance: A novel framework

Remote sensing image synthesis tasks aim at generating high-quality and realistic images based on given conditional inputs. Compared to existing methods that rely on semantic maps or text descriptions derived from satellite images, synthesizing satellite images from street views offers a more practical approach for generating satellite imagery of specific locations. Although diffusion models have shown great potential in recent satellite image synthesis research, current studies still face challenges such as the lack of fine-grained spatial text description datasets and issues with the realism and consistency of synthesis results. To address this, we first constructed Street2Sat-Text, a novel dataset aligning street-view images, satellite images, and text descriptions. By integrating visual language models (VLMs) and bird’s-eye view (BEV) transformation, it captures fine-grained spatial text from street-view images to guide accurate satellite image synthesis. We also extended our scalable text generation pipeline to CVACT, resulting in the multimodal CVACT-Text dataset. In addition, we propose a novel street-to-satellite image synthesis network that generates realistic and consistent satellite images from BEV images and spatial text, achieving state-of-the-art Fréchet inception distance (FID) and structural similarity index (SSIM) performance. Furthermore, we validated the effectiveness of our algorithm across several practical tasks such as disaster emergency response, historical high-resolution satellite image synthesis, and cross-view retrieval. Our work addresses the lack of fine-grained spatial text datasets and proposes an effective method for satellite image synthesis, demonstrating practical value across multiple downstream applications.