Semantic abstractions in the multimedia domain

Information searching by exactly matching content is traditionally a strong point of machine searching; this is not, however, how human memory works and is rarely satisfactory for advanced retrieval tasks in any domain-multimedia in particular, where the presentational aspects can be equally important to the semantic information content. A combined abstraction of the conceptual and presentational characteristics of multimedia applications, leading on the one hand to their conceptual structure (with classic semantics of the real-world modeled by entities, relationships, and attributes) and on the other to the presentational structure (including media type, logical structure, temporal synchronization, spatial (on the screen) "synchronization" and interactive behavior) is developed in this paper. Multimedia applications are construed as consisting of "Presentational Units:" elementary (a media object with play duration and screen position), and composite (recursive structures of PUs in the temporal, spatial, and logical dimensions). The fundamental concept introduced is that of Semantic Multimedia Abstractions (SMA): qualitative abstract descriptions of multimedia applications in terms of their conceptual and presentational properties at an adjustable level of abstraction. SMAs, which could be viewed as metadata, form an abstract space to be queried. A detailed study of possible abstractions (from multimedia applications to SMAs and SMA-to-SMA), a definition and query language for Semantic Multimedia Abstractions (SMA-L) and the corresponding SMA model (equivalent to extended OMT), as well as an implementation of a system capable of wrapping the presentational structure of XML-based documents complete this work, whose contribution lays in the classically fruitful boundary between AI, software engineering, and database research.