Inferring semantic concepts for video indexing and retrieval

This paper proposes a novel probabilistic framework for semantic indexing and retrieval in digital video. The components of the framework are probabilistic multimedia objects (multijects) and a network of such objects (multinet). The main contribution is a Bayesian multinet which enhances the detection performance of individual multijects and supports inference of concepts that are not observed directly in the multiple media. This inference is based on their relation with observable concepts. We develop multijects for detecting sites (locations) in video and integrate the multijects using a multinet in the form of a Bayesian network. We also use the site multijects and the multinet to infer the presence of the multiject Outdoor which has no direct support in media features.