A concept-based video retrieval model with spatio-temporal fuzzy triples

The drawback of conventional content-based video retrieval systems is that they cannot retrieve video shots semantically related with the user's intention. One of the reasons for this is that extracting semantics from the visual features and spatial structures of video streams is extremely difficult. To remedy this drawback, we propose a new video retrieval model based on fuzzy triples. In this model, the fuzzy triples are used for indexing video shots and formulating queries. Moreover, they are also used to express a set of active rules where the spatio-temporal semantics of the video streams is encoded. The fuzzy triples and the rules define generic spatio-temporal patterns in video streams by specifying the relative spatial relationships between salient objects and their changes with respect to time. Capturing such semantics entails concept-based video retrieval. For this, the fuzzy triples employ a fuzzy thesaurus and inexact matching. The inexact matching is needed when determining a direction from an angle formed counter-clockwise between two salient objects in a video shot. The novelty of the proposed model is that it can provide a concept-based video retrieval functionality which is conceptually simple and hence easy to construct.