The Perceptive Workbench: toward spontaneous and natural interaction in semi-immersive virtual environments

The Perceptive Workbench enables a spontaneous, natural and unimpeded interface between the physical and virtual worlds. It uses vision-based methods for interaction that eliminate the need for wired input devices and wired tracking. Objects are recognized and tracked when placed on the display surface. Through the use of multiple light sources, the object's 3D shape can be captured and inserted into the virtual interface. This ability permits spontaneity since either preloaded objects or those objects selected on-the-spot by the user can become physical icons. Integrated into the same vision-based interface is the ability to identify 3D hand position, pointing direction and sweeping arm gestures. Such gestures can enhance selection, manipulation and navigation tasks. In this paper, the Perceptive Workbench is used for augmented reality gaming and terrain navigation applications, which demonstrate the utility and capability of the interface.