Prioritized Experience Replay Method Based on TD Error and Reducible Loss

In the swiftly advancing realm of artificial intelligence, reinforcement learning plays a pivotal role. The experience replay method is the key to enable the RL method to achieve significant advancements, by using a dataset of past experiences to train a policy or a value function. However, the data acquired by RL agents often remains constrained. Posing challenges in leveraging it effectively, with even minor neural networks susceptible to overfitting during training. In order to improve the efficiency of the experience replay method, we propose a method for prioritizing samples based on Temporal Difference error and Reducible Loss (TDRl). By integrating Reducible Loss with Temporal Difference error, we can prioritize samples conducive to effective learning, while mitigating the influence of noise or stochasticity-induced samples. We apply TDRl method to DDQN algorithm and show that our method is more robust than vanilla PER method. We contend that the TDRl method holds the potential to broaden the scope of replay-based RL algorithms.