A framework for low level analysis and synthesis to support high level authoring of multimedia documents

We have designed and developed a hierarchical framework for composition of multimedia documents. This framework forms the basis of an important analysis technique for developing high level authoring tools. From the perspective of aiding the task of multimedia production, the emphasis has been placed on the generality and uniformity of the framework and on support for new techniques to improve the authoring environment. Specifically, this framework supports flexible synchronization and flow control as well as a range of interactive, continuous, and discrete media in a uniform fashion. We adopt the hierarchical composition approach in order to support a style of analysis similar to the semantic analysis performed in programming language translation. This analysis capability provides a robust base upon which a number of potential high-level authoring facilities can be built including resource analysis for conflict detection and resolution, interactive specification and editing of the temporal relationships among multiple media materials, and automatic synthesis of candidate solutions for detected problems. We examine various important aspects of this framework and describe how the potential high level authoring tools can be implemented using its analysis capability.