Temporal-Spatial Recommendation for Caching at Base Stations via Deep Reinforcement Learning

Proactive caching at the base station (BS) is a promising way to leverage the user-behavior-related information to boost network throughput and improve user experience. However, the gain of caching at the mobile edge highly depends on random user behavior and is largely compromised by the uncertainty in predicting behavior-related information. First, the local file popularity in each cell may not be skewed. Second, the local file popularity varies quickly due to user mobility even if the lifetime of each file is long. Furthermore, considering the small population of users that initiate requests in each cell, the local popularity in the next cache update period is not easy to predict accurately, because users may not request their interested files in this period, despite that the popularity can be indirectly obtained by predicting the mobility and preference of each individual user in a cell. To address such issue, in this paper, we integrate recommendation with caching at BS, aiming at improving cache efficiency whereas not violating user preference. In particular, we propose a temporal-spatial recommendation policy, which can guide mobile users to request their preferred files in proper time and place, so as to make local popularity peakier. We do not assume that the user preference, the impact of the recommendation on request probability, and the mobility pattern are known. Hence, we resort to deep reinforcement learning to optimize recommendation and caching policy. To deal with the difficulty in predicting local popularity in the next cache replacement period, we model the user preference and request probability with Bernoulli mixture distribution and hence can estimate them separately. The simulation results demonstrate that the proposed policy can reduce the cache miss number, compared to the policies without any recommendation and without temporal-spatial recommendation.