A reconfigurable architecture for multimodal and collaborative interactions in Virtual Environments

Many studies have been carried out on multimodal and collaborative systems in VR. Although these two aspects are usually studied separately, they share interesting similarities. This paper focuses on the reconfigurable aspect and the implementation of a multimodal and collaborative supervisor for Virtual Environments (VEs). The aim of this supervisor is to ensure the merge of pieces of information from VR devices in order to control immersive multi-user applications through the main communication and sensorimotor channels of humans. Beyond the architectural aspect, we give indications on the modularity and the genericity of our system, implemented in C++, which could be embedded into different VR platforms. Moreover, its XML-based configuration system allows it to be easily applicable to many different contexts. The reconfigurable features are then illustrated via two scenarios: a cognitive oriented assembly task with single user multimodal interactions, and an industrial assembly task with multimodal and collaborative interactions in a co-located multi-user environment.