Content-Based Video Retrieval Using Video Ontology

In this paper; we aim to efficiently retrieve various kinds of events (e.g. conversation, battle, run/walk and so on) from a video archive. To this end, we construct a "video ontology" which is a formal and explicit specification of events. Specifically, an event is modeled to have 4 dimensions of semantic contents (i. e. Action, Location, Time and Shooting technique). For retrieving such events, concepts in 4 dimensions need to be automatically detected. So, we conduct "video data mining" to extract "semantic patterns" from videos. Here, a semantic pattern is a combination of low-level features (e.g. color; motion and audio) associated with events of a certain kind. Thus, semantic patterns can be used to characterize concepts in 4 dimensions of semantic contents. Furthermore, we refine the video ontology by extracting new semantic patterns from subspaces of videos, which cannot be retrieved by previously extracted patterns. Finally, we classify each event into video genres which potentially contain this event. It is useful for limiting video genres from which events of user's interest are retrieved.