SmartTalk: A Learning-Based Framework for Natural Human-Robot Interaction

This paper presents a learning-based frameworknamed SmartTalk for natural-language human-robot interaction (HRI). The primary goal of this framework is to enable non-expert users to control and program a mobile robot using natural language commands. SmartTalk is modality-agnostic, and is capable of integrating with both speech and non-speech (e.g., gesture-based) communication. Initially, robots using this mechanism are equipped with a limited vocabulary of primitive commands and functionality, however, through extended use andinteraction, the robots are able to learn new commands and adapt to user's behaviors and habits. This makes the proposed framework highly desirable for long-term deployment in a variety of HRI tasks. We present the design of this framework and experimental data on a number of realistic scenarios to evaluate its performance. A qualitative experiment on a robotic platform is also presented.