Deep Predictive Network for Inference and Dynamic Optimization of Task Goals during Human-Robot Collaboration

Collaborative robots are expected to generate actions by inferring a task goal from an environmental situation while performing dynamic optimization of the inferred goal based on a human partner's behavior. The objective of this study was to develop a computational framework that enables collaborative robots to learn this inference ability. For this, we propose a deep-learning based framework consisting of (1) a goal-inference network, (2) a goal-recognition network, and (3) an action- generation network. The goal-inference network is jointly trained with the goal-recognition network to infer a latent goal only from the initial image of a task space. The action-generation network generates a prediction about the visuomotor state from its current state and the inferred latent goal. During action-generation, the inferred latent goal is dynamically optimized to minimize visual prediction errors in order to adapt to changes in the human partner's goal. To evaluate this framework, we conducted an experiment on a collaborative object arrangement task. In the task, a human was allowed to change its goal during collaboration and a robot was required to adapt to such a situational change by inferring the human's goal. The experimental results demonstrate that the robot using the developed framework realized a successful collaboration.