A Context-Based Whole Video Retrieval System with Dynamic Video Stream Analysis Mechanisms

With the rapid expansion of fictional video contents on WWW, it is important to develop video retrieval mechanisms for fictional video contents, such as TV drama, animations, and movies. This paper proposes a context-based video retrieval system for retrieving an entire video stream, not individual scenes or fragments. The unique feature of this system is a SQL-like contextual multimedia query language, which allows users to express and characterize their queries by using video clips as contexts. It extracts emotive contexts of visual elements from video files by analyzing the color-emotions hidden within video content using methods developed during color psychology research. The system's capabilities enables new ways of querying artistic, visually rich and dynamic video content - something which is not possible by using just a few words. This is done by inspecting the dynamic and time-evolving contexts of video data. A prototype system has been implemented which is applicable to performing queries against television content. This paper clarifies the feasibility of the system by demonstrating the results of several experiments.