Simulation-Based Robot Skill Learning System Using PPO and Cauchy-Distribution-Based Reward Design

In recent years, the application of robots in the industrial sector has been steadily increasing, leading to a gradual rise in the workload of robot development. However, the cost of conducting physical experiments for functional development tailored to different scenarios is high and time-consuming. In this paper, we propose a robot skill learning structure to enable robot to learn specific skills automatically without user programming in a simulation scenario. The proposed structure consists of a skill learning platform and a skill learning toolbox. The skill learning platform is designed to construct simulation scenarios for skill learning for various types of robots. The skill learning toolbox is designed to provides necessary tools for simulation and skill learning using Docker. The Proximal Policy Optimization (PPO) algorithm is served as skill learning algorithm. A Cauchy-distribution-based reward design method is proposed to select the optimal influence coefficient of each reward term and then to obtain the highest reward. Under this robot skill learning structure, robot can successfully learn a skill in a simulation scenario through our experiments. Additionally, using this skill learning structure can reduce the time of development, and increases the reusability rate of the skill.