Semantic Object Understanding in UAV Operations Using Vision-Language Models

The application of vision-language (VL) models in Internet of Drones (IoD) frameworks is a groundbreaking solution for semantic comprehension of aerial tasks. In contrast to conventional computer vision techniques that utilize static-class detectors, VL models facilitate dynamic interpretation of natural language commands, and hence allow adaptive and context-sensitive object detection. In this paper, we introduce an extensible and scalable model in which unmanned aerial vehicles (UAVs) send images and telemetry information to a cloud-processing entity. A vision-language model is applied to the incoming visual data, and telemetry is employed to increase spatial and contextual awareness. The outputs are fused and maintained in a structured JSON format with added metadata, thereby enabling real-time as well as historical insights. The architecture of the system integrates RESTful APIs, contextual fusion engines, and prompt configuration modules to enable versatile deployment across varying operating environments. Experimental results indicate a mean average precision of 0.87 across diverse object types and an end-to-end inference latency of 670 milliseconds, confirming the system’s ability to deliver accurate and prompt semantic insights. Experimental findings show that the proposed model enables precise, prompt-driven perception and lays the groundwork for intelligent aerial data analysis for urban surveillance, emergency response, infrastructure monitoring.