Multistage Synergistic Aggregation Network for Remote Sensing Visual Grounding

Visual grounding (VG) has a broad application prospect in the field of remote sensing (RS). Current state-of-the-art methods predominantly are based on the transformer architecture, utilizing multihead self-attention in multimodal encoders to integrate visual and textual features. However, they typically rely on a single fusion approach, which may limit the model’s capacity to learn intricate correlations between textual semantics and visual information. Moreover, they did not establish a direct dependency between features and bounding box representations, thereby restricting the fusion features to conventional object detection paradigm. Consequently, the interactions between regression results and encoded features are constrained. To address these limitations, a generative paradigm is harnessed to directly generate discrete coordinates sequence in an autoregressive manner, which explores the interaction between direct regression features and encoded multimodal features. Meanwhile, a novel multistage synergistic aggregation module (MSAM) is proposed to facilitate the acquisition of multimodal features at multiple scales by effectively aggregating visual and textual contexts, enhancing the overall performance. In this work, we validate our framework on the DIOR-RSVG dataset and conduct a comparative analysis with the existing methods, achieving a noteworthy improvement in accuracy. The proposed approach presents a promising direction for advancing VG techniques in the context of RS applications. The related code and weights are available at https://github.com/waynamigo/MSAM.