Tell me when and why to do it! Run-time planner model updates via natural language instruction

Robots are currently being used in and developed for critical HRI applications such as search and rescue. In these scenarios, humans operating under changeable and high-stress conditions must communicate effectively with autonomous agents, necessitating that such agents be able to respond quickly and effectively to rapidly-changing conditions and expectations. We demonstrate a robot planner that is able to utilize new information, specifically information originating in spoken input produced by human operators. We show that the robot is able to learn the pre- and postconditions of previously-unknown action sequences from natural language constructions, and immediately update (1) its knowledge of the current state of the environment, and (2) its underlying world model, in order to produce new and updated plans that are consistent with this new information. While we demonstrate in detail the robot's successful operation with a specific example, we also discuss the dialogue module's inherent scalability, and investigate how well the robot is able to respond to natural language commands from untrained users.