A Reinforcement Learning Sampling Optimization Method Based on Training Value

Reinforcement learning (RL) has been widely used in various real-world applications, such as robotics, game playing, and finance. However, traditional RL algorithms exhibit limitations in training efficiency and performance. In this paper, we propose a novel RL optimization algorithm based on training value. The proposed algorithm makes use of the training value metric to improve experience sampling and utilization, accelerating the training process and improving the agent's performance in the task. The algorithm calculates training value by considering the TD error of experience data, and prioritizes experience with high training value during training. We demonstrate the effectiveness of the proposed algorithm through experiments on multiple Atari game environments. Compared to traditional prioritized experience replay (PER) and baseline DQN algorithms, the proposed algorithm achieves comparable or better average reward in most environments and significantly reduces the training time. These results suggest that the proposed training value-based optimization method can effectively improve the performance of RL algorithms and can be a promising approach for future RL research.