Dobby: A Conversational Service Robot Driven by GPT-4

This work introduces a robotics platform which comprehensively integrates multi-step action execution, natural language understanding, and memory to interactively perform service tasks in accordance with variable needs and intentions of users. The proposed architecture is built around an AI agent, derived from GPT-4, which is embedded in an embodied system. Our approach utilizes semantic matching, plan validation, and state messages to ground the agent in the physical world, enabling a seamless merger between communication and behavior. We demonstrate the advantages of this system with an HRI study comparing mobile robots with and without conversational AI capabilities in a free-form tour-guide scenario. The increased adaptability of the system is measured along five dimensions: flexible task planning, interactive exploration of information, emotional-friendliness, personalization, and increased overall user satisfaction.