Natural Language Generation for Socially Competent Task-Oriented Agent

Conversational Artificial Intelligence has been used in many industrial use cases, especially in customer service. While emotion is often associated with chit-chat and open-domain interactions, it is still a core aspect of human interaction. Though arguably less frequent, a customer’s emotional and social displays are very meaningful in official task-oriented dialogue. Hence why it is crucial for vocal dialogue systems to learn how to properly detect and adapt their behaviour to respond to this social context. This context can be represented by textual and semantic content, but it can also be multimodal, using prosody elements extracted from speech analysis. This research aims to investigate a method to take these social and emotional components for conditional generation, in order to develop a socially competent, task-oriented conversational agent. In this Doctoral Consortium paper, I introduce my PhD subject, its scope, and the main research questions we tackle. I give an overview of the related works, before presenting the methodology, the work done, and the avenues we intend on exploring in order to respond to our established research objectives.