Intelligent hybrid video retrieval system supporting spatio-temporal correlation, similarity retrieval

We suggest the intelligent hybrid video retrieval system (IHVRS) which provides content-based querying integrating feature-based queries, annotation-based queries, and similarity retrieval. It supports the approximate query results by using query reformulation in the case that the result of the query does not exist. The IHVRS divides a set of video into video documents, sequences, scenes and objects, and suggests the three layered hybrid object-oriented metadata model (THOMM) to model metadata. The THOMM is composed of a raw-data layer for the physical video stream, a metadata layer to support the annotation-based retrieval, feature-based retrieval, and similarity retrieval and a semantic layer to reform the query. Based on this model, we suggest a video query language which makes annotation-based queries and feature-based queries based on color, spatial, temporal and spatio-temporal correlation and similar queries possible and an intelligent video query processor (IVQP) to process the query. For the case of similarity queries on a given scene or object, we present the formula expressing degree of similarity based on color, spatial, and temporal order. If there is no query result, then it carries out the query reformulation process which finds the possible attributes to relax the query and automatically reforms the query by using the knowledge from the semantic layer. The suggested system is implemented by using Visual C++, ActiveX and ORACLE. The IHVRS can automatically input features of an object, color and spatial location, without image processing.