Multiple Target Prediction for Deep Reinforcement Learning

This paper presents the multiple target prediction for model-free deep reinforcement learning. Traditionally, the learning algorithm based on deep Q network (DQN) suffers from slow convergence in learning process which usually constrains the system performance. To speed up the learning process, this study incorporates the prediction network and the auxiliary replay memory in DQN which allows us to predict multiple target values over different actions rather than relying on a single target value from an action. The prediction network is trained in a sequential manner by using the samples from replay memory while the auxiliary replay memory is introduced to store the states and the predicted targets which are related to individual possible actions that are taken. With these two additional components functioned in DQN algorithm, the resulting approach can efficiently predict Q values and rewards to train an agent with comprehensive target values. The experiments on different tasks demonstrate the merit of multiple target prediction in reinforcement learning.