Once More Unto the Breach: Co-evolving a robot and its simulator

One of the major challenges facing evolutionary robotics is crossing the reality gap: How to transfer evolved controllers from simulated robots to real robots while maintaining the behavior observed in simulation. Most attempts to cross the reality gap have either applied massive amounts of noise to the simulation, or conducted most or all of the evolution onboard the physical robot, an approach that can be prohibitively costly or slow. In this paper we present a new co-evolutionary approach, which we call the estimation-exploration algorithm. The algorithm automatically adapts the robot simulator using behavior of the target robot, and adapts the behavior of the robot using the robot simulator. This approach has four benefits: the process of simulator and controller evolution is automatic; it requires a minimum of hardware trials on the target robot; it could be used in conjunction with other approaches to automated behavior transferal from simulation to reality; and the algorithm itself is generalizable to other problem domains. Using this approach we demonstrate a reduction of three orders of magnitude in the number of evaluations on a target robot (thousands compared to only five).