Robot Confrontation Based On Genetic Fuzzy System Guided Deep Deterministic Policy Gradient Algorithm

Robot confrontation problem is a kind of zero-sum games, whose analytical solution is the solution of Hamilton-Jacobi-Isaacs equations that is generally unable to derive. In order to approximate the solution, a novel genetic fuzzy system guided deep deterministic policy gradient algorithm(GFS-DDPG) is proposed. As well known, because of utilizing the deep neural network to represent the critic and the actor, the DDPG algorithm has an ability to learn the optimal solution for any system. However, the learning process is slow and sensitive to the super parameters due to the poor exploration. While the genetic fuzzy system learns fast at the beginning stages of learning for its strong exploration in the parameters space. The GFS-DDPG algorithm combines their advantages. At the beginning of each episode, a population that is initially trained by GFS is used to explore the environment and collect data, then the learning process of DDPG is adapted to update actor-critic networks through gradient descent. Experimental results show that the proposed algorithm performs better than GFS and DDPG both in the intelligent level and the learning efficiency.