Multivalent documents: inducing structure and behaviors in online digital documents

Digital documents have advanced only slightly from their non-digital ancestors, incorporating a few such innovations as hypertext links and full text searching. Even the much heralded advent of multimedia, just as embedded images and video clips, are rather straightforward digital analogs of books and film. We believe that computer technology makes possible far more interesting and complex digital documents, and propose a new general paradigm that regards complex documents as "multivalent documents" comprising multiple "layers" of distinct but intimately related content. Small, dynamically-loaded program objects, or "behaviors", activate the content and work in concert with each other and layers of content to support arbitrarily specialized document types. Behaviors bind together the disparate pieces of a multivalent document to present the user with a single unified conceptual document. Examples of the diverse functionality in multivalent documents include: "OCR select and paste", where the user describes a geometric region on the scanned image of a printed page and the corresponding text characters are copied out; video subtitling, which aligns a video clip with the script and language translations so that on the one hand the playing video can be presented simultaneously in multiple languages, and on the other hand the video can be searched with text-based techniques; geographic information system (GIS) visualizations that compose several types of data from multiple datasets; and distributed user annotations that augment and may transform the content of the conceptual document. In general, a document management infrastructure built around a multivalent perspective can provide an extensible, networked system that supports incremental addition of content, incremental addition of interaction with the user and with other components, reuse of content across behaviors, reuse of behaviors across types of documents, and efficient use of network bandwidth. Multivalent documents exploit digital technology to enable new, more sophisticated document interaction.