The Haptic Desktop: a novel 2D multimodal device

As computing gets embedded into our everyday lives and moves from PCs to wide varieties of devices, we need new and flexible ways to interact with technology. Recently, the appearance of multimodal systems has been recognized to offer richer user experiences by combining different natural input modes, i.e., speech, touch, pen, hand gestures, etc.. in a coordinated manner with multimedia system outputs. As a new generation of multimodal systems begins to define itself, one dominant theme is the integration and synchronization requirements for combining different modes strategically into whole systems; where we can distinguish basically three modalities: visual, auditive, and tactile (physiology of senses). In order to provide a broader range of signal integrations, actually limited by the conventional input devices of computers, it becomes necessary to ensure that these interfaces interact in a manner akin to the way human interacts in real environments. Therefore, The design and integration of a novel multimodal device developed at PERCRO is described. This multimodal device, named Haptic Desktop, has been designed to replace the conventional input devices used for accessing the resources of computers. Furthermore, to assure the usability of the interface so that operators can extract efficiently the principal features of each perceptual channel from the device, the graphical visualization has been integrated to the haptic system on a single desktop to satisfy the coherence and co-location designing issues.