Double Replay Buffer Mechanism for Deep Reinforcement Learning Algorithm

Deep Reinforcement Learning (DRL) has emerged as a highly promising area of research within the fields of artificial intelligence and machine learning. However, the conventional experience replay mechanism employed in DRL typically utilizes a uniform sampling method, which fails to account for the varying importance of different samples. This oversight can result in suboptimal sample utilization and low learning efficiency. To address these limitations, this paper proposes a double experience replay buffer mechanism. The proposed approach adaptively adjusts the experience pool ratio through the Q-learning algorithm and leverages the Upper Confidence Bound (UCB) method to calculate sample priorities. Furthermore, we design a hybrid sampling strategy to increase the selection rate of high-value samples, and enhance overall learning performance. To validate the effectiveness of this approach, comprehensive experiments were conducted across several Gym control tasks. The experimental results demonstrate the improvements in learning efficiency and performance.