Sim2Real Deep Reinforcement Learning of Compliance-based Robotic Assembly Operations

Reinforcement learning (RL) enables robots to learn goal-oriented behavior. In production processes with high variances, such as joining operations in end-of-line assembly, this is particularly interesting to save significant programming effort. Due to a large amount of required training data, simulative training is becoming increasingly important. In this paper, we present an approach to learn a contact-rich peg-in-hole assembly task utilizing deep reinforcement learning (DRL) and a compliant robot controller. The DRL-Agent learns directly in the Cartesian space (task space) and not in the joint space of the robot, to increase the robustness and efficiency of the algorithms. To further increase the robustness of the policy and to shorten training times, geometric limitations are imposed by introducing an admissible workspace using a trajectory generator. Furthermore, these limitations result in nearly identical behavior in the simulation and on the real robot, allowing the DRL training process to be purely simulative. The learned policy is experimentally investigated both in the simulation environment and on a real robot, to evaluate its transferability from simulation to reality (sim2real).