Development of a collaborative multimodal system with a shared sound display

Ubiquitous computing, or pervasive computing, is an increasing popular technique for developing new human-computer interfaces and computer applications. Computers watch what happens in a physical space where people are and serve them with meaningful information. Here, most of the systems depend on visual interaction, while there are some other interaction media. In this paper we present a collaborative multimodal system toward spatial presence and perceptual realism of sounds. A two-dimensional sound display called Sound Table is its central component, in which 16 speakers are mounted so that multiple users can have a sensation of “sounds are here and there”. A stick-shaped device is provided as a means for interaction with the system. The system recognizes manipulation of the device in a 3D space over Sound Table, and then gives users feedback through visual channel as well as auditory one.