Handling the environment and the users' constraints by rational dialoguing agents

In this paper we present part of our work concerning multimodal dialogue systems founded on rational agents. We introduce the logical framework of our theory of rational agency. We present how this formal framework enables the modelled agents to take into account various aspects of multimodal interaction, such as the physical state of the interaction scene and the physical and mental state of the involved agents. In particular, we propose a model of communicative actions, which defines multimodal dialogue acts. The preconditions of the act model are based on a communication model. These preconditions enable agents to reason on this model and adapt their behavior regarding the physical and mental constraints of the multimodal dialogue.