Q-Learning Based Delay-Aware Content Delivery in Cloud-Edge Cooperation Networks

As network mobile traffic grows rapidly, how to allocate various network resources to improve the efficiency of content distribution and quality of experience is an urgent problem to be solved for current delay-sensitive services. To efficiently reduce network delay, the idea of cloud-edge cooperation has recently drawn great attention and been explored. However, the joint allocation of caching, computing and transmission resources has been lack of in-depth study, and content popularity closely related to traffic distribution has been largely ignored in existing works. In this paper, we propose a Q-learning based delay-aware content delivery scheme in a three-layer cloud-edge cooperation network. In our solution, the queueing theory is used to formulate the content distribution process as a delay minimization problem, where a Q-learning policy is utilized to make optimal cache placement decisions to improve network delivery and delay. Simulation results demonstrate that our proposed model performs much better than the related popular solutions and close to its centralized alternative.