Using Diagrams for the Semantic Annotation of Multimedia

Currently there is much interest in the creation of welldefined multimedia annotations, especially in the context of the Semantic Web and MPEG standards. However, there is a lack of tools that support the creation of standards-based annotations. We introduce an approach that lets users create annotations using simple diagrams with unconstrained linking and labelling and an architecture for formally interpreting them in an interactive process that matches the user input with formal ontologies based on information from the ontologies itself and general background knowledge. The interpretations can be used for reasoning tasks, such as querying and validation, and to guide the user in the further knowledge creation process. We discuss the state of the art in related technologies, outline the design, architecture and algorithms, and describe the implemented prototype.