Propagating learned behaviors from a virtual agent to a physical robot in reinforcement learning

For a physical robot to acquire behaviors, it is important for it to learn in the physical environment. Since reinforcement learning requires large computation costs as well as a lot of time in the physical environment, most research has performed learning by simulation. However, this does not work well in the real world. Realizing reinforcement learning of a physical robot in a physical environment requires both an adaptation for the diversity of possible situations and a high-speed learning method that can learn from fewer trials. This paper describes cooperative reinforcement learning based on propagating the learned behaviors of a virtual agent to a physical robot in order to accelerate learning in a physical environment. The method consists of two parts: (1) preparation learning in a virtual environment to accelerate initial learning, which accounts for most of the learning cost; and, (2) refinement learning in a physical environment by using the virtual learning results as an initial behavior set of a physical robot. Experimental results are given for a ball-pushing task with the physical robot and a virtual agent.