AirSpatialBot: A Spatially Aware Aerial Agent for Fine-Grained Vehicle Attribute Recognition and Retrieval

Despite notable advancements in remote sensing (RS) vision-language models (VLMs), existing models often struggle with spatial understanding, limiting their effectiveness in real-world applications. To push the boundaries of VLMs in RS, we specifically address vehicle imagery captured by drones and introduce a spatially aware dataset AirSpatial, which comprises over 206k instructions and introduces two novel tasks: spatial grounding (SG) and spatial question answering (SQA). It is also the first RS grounding dataset to provide a 3-D bounding box (3DBB). To effectively leverage existing image understanding of VLMs to spatial domains, we adopt a two-stage training strategy comprising image understanding pretraining and spatial understanding fine-tuning. Utilizing this trained spatially aware VLM, we develop an aerial agent, AirSpatialBot, which is capable of fine-grained vehicle attribute recognition and retrieval. By dynamically integrating task planning, image understanding, spatial understanding, and task execution capabilities, AirSpatialBot adapts to diverse query requirements. Experimental results validate the effectiveness of our approach, revealing the spatial limitations of existing VLMs while providing valuable insights. The model, code, and datasets will be released at https://github.com/VisionXLab/AirSpatialBot