A Reinforcement Learning Based Smart Cache Strategy for Cache-Aided Ultra-Dense Network

The integration of caching and ultra-dense network (UDN) can not only improve the efficiency of content retrieval by reducing duplicate content transmissions but also improve the network throughput and system energy efficiency (EE) of the UDN. In this paper, we focus on energy consumption aspects of cache-aided UDN (CUDN) and develop a novel caching strategy to improve the system EE. Different from the existing researches, we consider a more realistic scenario where the popularity of the cache content is dynamic and unknown. The proposed caching strategy is a deep reinforcement learning (DRL)-based approach that uses a deep Q-network to approximate the Q action-value function. We optimize the structure and corresponding parameters of the deep Q-network according to the latest findings in the field of DRL and deep learning (DL) to improve the performance of this caching strategy. The simulation results show that the performance in terms of EE of the CUDN can be significantly improved by using our proposed caching strategy.