Reinforcement Learning with Evolutionary Computation to Policy Search for Autonomous Navigation

Reinforcement learning has good applications for autonomous navigation in unknown and complex environments. Traditional reinforcement learning methods with the actor-critic framework sometimes will fall into a local optimum because of the complexity of the loss function. Meanwhile, evolutionary computation(EC) is a type of black box optimization algorithm, which has good robustness in policy search but lower sampling efficiency. In order to address the challenge, we introduce an algorithm that combines evolutionary computation with reinforcement learning into navigation intuitively. The parameters of actor neural network are listed as individual characteristics. Each individual represents a policy network. At the end of each episode, individuals with higher fitness function value are selected to the next generation. Other individuals update a certain number of steps through the critic network with shared replay buffer and then move into the next generation. Simulation results demonstrate the effectiveness and feasibility of this algorithm on navigation.