Learning how to drive using DDPG Algorithm with Double Experience Buffer Priority Sampling

The driving policy based on the DDPG algorithm has the defects of convergence difficulty and poor anti-interference ability. An important reason for these defects is that the random experience replay mechanism cannot reasonably utilize experience collected by agents. It can neither use different experiences for targeted training in different training stages, nor can it pick out high-quality experience to speed up policy convergence. This paper proposes a double experience buffer priority sampling mechanism, which can distinguish positive experience and negative experience, and replay these experiences according to the needs of the current training stage, and can also select high-quality experience for priority replay. The results of simulation experiments show that our method not only increases the convergence speed of the policy, but also improves the driving performance in the new environment.