Adding intelligence to the interface

Virtual reality has made computer interfaces more intuitive but not more intelligent. This paper shows how a rule-based expert system can be combined with multimodal input and natural language techniques to add intelligence to the interface. In this way, virtual worlds can be built which recognize and respond intelligently to user actions. Coupling voice, gesture, body position and context together with an expert system creates an interface more powerful than any of these elements alone; one which establishes a true two-way dialog between human and machine. To illustrate these techniques, we present two prototype systems that allow intuitive multimodal communication within intelligent virtual environments.