AI Video Retrieval: A Semantic Search & Timestamp Alignment System

Traditional video search systems depend on keywords and manual tagging, limiting their ability to retrieve video segments that semantically match user queries. While recent studies explore image captioning and audio-based retrieval, real-time, scalable frameworks with multimodal indexing remain limited. This paper presents an AI Video Retrieval (AIVR) system that overcomes these challenges using deep learning models—Whisper-Timestamped for speech transcription, Blip for frame captioning, and Sentence Transformers for embedding generation. Integrated through Django and indexed with TXTAI, the system supports real-time video uploads, transcribes audio, captions sampled frames, and structures multimodal data in JSON for efficient semantic search. It retrieves accurate video segments based on natural language queries and demonstrates improved retrieval relevance, timestamp precision, and usability over traditional methods. The framework is extendable with OCR, object detection, and action recognition, and has practical applications in education, media, and surveillance.