Integrating Communication with Interaction: Computer Vision Challenges for Interactive and Intelligent Environments

Interactive, Intelligent Environments involve a convergence of various research themes, including high-fidelity visualization, communication, gestural expression, and virtualized reality systems. Recent advances in real-time acquisition, transmission, and rendering of multimodal data (e.g. audio, video, haptic) allow for the synthesis of significantly improved perceptual representations of a virtual or real (e.g. remote) environment than were previously possible. Furthermore, increased computational power permits the synthesis of a rich responsive media space that responds to a large number of participants engaged in a complex, expressive activity. Unfortunately, current systems tend to concentrate almost exclusively on one aspect or the other, supporting the representation and interaction with a virtual world, or supporting distributed human communication, but never both. The ideal interactive intelligent environment is one that permits effective distributed human-human communication among large numbers of participants at multiple locations, simultaneously with data visualization capabilities and interaction with dynamic, synthetic objects. A significant challenge for the next generation of such environments is to develop the necessary physical infrastructures and software architectures that combine these capabilities appropriately.