Language-driven nonverbal communication in a bilingual conversational agent

This paper describes an animated conversational agent called Kare which integrates a talking head interface with a linguistically motivated human-machine dialogue system. The agent has a range of nonverbal behaviors, which involve a mixture of machine vision, computer animation and natural language processing techniques. The system's architecture couples the agent's nonverbal communicative processes very tightly to its model of verbal interaction. We discuss several consequences of this architecture, in particular the ability to use different nonverbal dialogue management signals when speaking different languages.