Distributed cache replacement for caching-enable base stations in cellular networks

Distributive service data storage at the caching-enabled base stations (BSs) can reduce the traffic load in future cellular networks. Taking the limited caching space into account, it is necessary for the BSs to adjust their caching data based on service popularity in order to achieve better caching efficiency. In this paper, we investigate the cache replacement strategy for BSs to minimize the transmission cost between BSs in cellular networks. The cache replacement problem is modelled as a Markov Decision Process (MDP). Without extra information exchange about caching data between the BSs, we propose a distributed cache replacement strategy based on Q-learning. Especially, we calculate the transmission cost for possible cache replacement actions according to the previous data request and transmission between BSs. The convergence of the proposed distributed cache replacement strategy is proved by sequential stage game model. Simulation results verify the convergence of the proposed cache replacement strategy and show its performance gain compared to conventional strategies.