Deep Q-Learning Aided Energy-Efficient Caching and Transmission for Adaptive Bitrate Video Streaming Over Dynamic Cellular Networks

Adaptive bitrate video streaming (ABRVS) and edge caching are two techniques that hold the potential to improve user-perceived video viewing experience. In this paper, we investigate the content caching, transcoding and transmission for ABRVS in cache-enabled cellular networks. Considering the dynamic characteristics of video popularity distribution and wireless network environment, to improve energy efficiency and minimize system energy consumption, we begin by formulating a long-term optimization problem that focuses on both video caching and user association (UA). The problem is then transformed into a Markov decision process (MDP), which is solved by designing a deep Q-learning network (DQN)-based algorithm. Using this algorithm, we can obtain the optimal video caching and UA solutions. Since the action space of the MDP is huge, to cope with the “curse of dimensionality”, linear approximation is integrated into the designed algorithm. Finally, the proposed algorithm’s convergence and effectiveness in reducing long-term system energy consumption are demonstrated through extensive simulations.