Dual-Stream Spatially Aware Transformer for Remote Sensing Image Captioning

Remote sensing image captioning (RSIC) aims to generate semantically rich and syntactically accurate descriptions for remote sensing images (RSIs). However, due to the complex spatial layouts, occlusions, and overlapping objects in such images, caption generation is often challenged by semantic ambiguity. To address these issues, we propose a novel dual-stream spatially aware transformer (DSAT), which explicitly models both global and local spatial relationships to enhance spatial understanding. Specifically, DSAT introduces a dual-stream feature interaction module that extracts grid-level global features and region-level object features, and further enhances their respective spatial dependencies through multibranch convolution and a graph attention network. In addition, we design a spatially aware attention mechanism that encodes relative spatial relationships into the Transformer, allowing the model to better capture object distribution patterns and geometric relationships. Extensive experiments conducted on three benchmark datasets, namely Sydney-Captions, UCM-Captions, and remote sensing image description, highlight the superior performance of DSAT. The proposed method achieves impressive CIDEr scores of 338.59%, 450.93%, and 275.36% on these datasets, respectively, demonstrating its effectiveness in generating high-quality captions for RSIs.