Brain Inspired Episodic Memory Deep Q-Networks for Sparse Reward

Although deep reinforcement learning has achieved great success in recent years, it still suffers from slow convergence, low sample efficiency, and large computational resources due to the existence of reward sparsity in many decision problems. Therefore, exploring more effective algorithms that can cope with sparse rewards is of great importance. Episodic memory reinforcement learning has received a lot of attention for its ability to collect past dominant strategies, which can serve as successful experiences to efficiently guide agents in sparse reward environments. In this paper, the episodic memory deep Q-networks, which incorporates the fast convergence property of episodic memory into neural networks, is employed to solve decision making problem with sparse rewards. Atari games are the test beds. Experiments show that the episodic memory deep Q-networks outperforms the deep Q-networks and the prioritized experience replay, which demonstrates the sample efficiency and the effectiveness of episodic memory deep Q-networks for the sparse reward problem.