The dialogue facilitator bot: Reflections on design and evaluation

With the growing popularity of chatbots, there is also a need for targeted evaluation criteria and metrics to assess their efficacy. Multiparty chatbots are designed to communicate with multiple users in a group, functioning as tools that support group interaction. Facilitator bots are a specific type of multiparty bots that assume the role of dialogue facilitator, to motivate and guide meaningful conversation in creative, professional, education or other settings. In this work we present the findings from a literature review of existing approaches to the evaluation of multiparty bots, informed also by the wider domain of chatbot design and evaluation. We examine and reflect upon the function and importance of different aspects of evaluation in relation to the specific chatbot objectives. These aspects include general ones, like user experience and chatbot personality, as well as more specific ones for multiparty bots, such as the function of the bot as dialogue facilitator and the quality of the dialogue between the chatbot and the users themselves.