Integration of eye-gaze, voice and manual response in multimodal user interface

This paper reports the utility of eye-gaze, voice and manual response in the design of multimodal user interface. A device and application-independent model (VisualMan) of selection and manipulation was developed and validated in 3D cube manipulation task. The multimodal inputs are integrated in a prototype interface based on priority of modalities and interaction context. The implications of the model for virtual reality interface are discussed and a simple virtual environment using present multimodal user interface model is proposed.