Advanced Object Detection Using Semantic and Temporal Query-Based Frame Retrieval in Surveillance Videos

With the fast expansion of surveillance infrastructure, there is an increasing demand for intelligent systems capable of comprehending and extracting useful information from massive amounts of video footage. This research introduces a novel surveillance architecture that integrates real-time object detection, semantic scene interpretation, and natural language querying to allow for efficient frame-level analysis. The system detects objects with high accuracy using the YOLOv8 model and then extracts spatial relationships to contextualize item interactions inside frames. Our approach includes user-defined parameters that enable filtering based on object type, description, and time range. Beyond detection, our system dynamically generates spatial and proximity relationships between identified items, as well as saves metadata for efficient video frame querying and retrieval. By integrating generative AI, we generate natural language summaries to enhance interpretability from the relationships generated. A Retrieval-Augmented Generation (RAG) model is employed in the final step to match generated summaries with user-provided descriptions. By this method, summaries that best describe user-defined descriptions are analysed, enabling precise retrieval of frame that best match the query. Our implementation demonstrates significant improvements in surveillance analytics by enhancing detection accuracy, retrieval speed, and interpretability of video content.