The Office of the Past: Document Discovery and Tracking from Video

This paper presents an approach for reconstructing the physical state of documents and other objects on a desk over time using an overhead video camera. The history of the desktop is subsequently analyzed to enable a range of interesting queries. A desktop object is discovered when it first moves, and is tracked through the image sequence using change detection and motion segmentation techniques. The space-time structure of the scene is represented by a sequence of graphs where an "event", i.e., a change in the state of the scene, is modeled as a transition between subsequent graphs. We present two prototype applications that are based on these desktop reconstruction techniques. The first one allows the user to browse the structure and history of the desktop, and query documents of interest. The second application is a virtual desktop interface where the user can virtually manipulate the document stacks with mouse interaction.