Edge Caching at Base Stations With Device-to-Device Offloading

Content caching at network edge nodes, such as base stations (BSs) and user equipments (UEs), can significantly reduce the traffic load in future cellular networks. Considering the limited caching space, the contents cached at BSs should be selected carefully for improving caching efficiency. In this paper, we study the edge caching at BSs to minimize the transmission cost by considering traffic offloading via the device-to-device (D2D) communications. The traffic offloading reduces the traffic via cellular transmission and thus changes the utility achieved by content caching at BSs. We model the edge caching problem as a Markov decision process and propose a distributed cache replacement strategy based on Q-learning. The proposed strategy further needs the calculations of the following two key parameters: 1) To describe the effect of D2D offloading to the cellular traffic, we define the cellular serving ratio, which is calculated by the iterative maximum weighted independent sets problem for static networks and the stochastic geometry for high-dynamic networks; 2) The cache replacement rewards are calculated by analyzing the relationship between the requested and cached amounts of content data, which are obtained from the messages of the previous data request and transmissions, ignoring any extra information exchange between the BSs. Furthermore, the convergence of the proposed distributed cache replacement strategy is proved by the sequential stage game model. Simulation results verify the convergence of the proposed cache replacement strategy and show its performance gain compared with conventional strategies.