Semi-automatically Configured Fission for Multimodal User Interfaces

Fission of several output modalities poses hard problems, and (semi-)automatically configuring it is even more difficult. However, it is important to address the latter in order to broaden the scope of providing user interfaces semi-automatically. Our approach starts from a high-level discourse model created by a human interaction designer. It is modality-independent,so a modality-annotated discourse model is semi-automatically generated. Based on it, our fission is semiautomatically configured. It currently supports output modalities graphical user interface, (canned) speech output, and a new modality that we call movement as communication. The latter involves movements of a semi-autonomous robot in 2D-space for reinforcing the communication of the other modalities.