Dueling Deep-Q-Network Based Delay-Aware Cache Update Policy for Mobile Users in Fog Radio Access Networks

Fog radio access networks (F-RANs) can effectively alleviate fronthaul loads and reduce content transmission delay by migrating cloud services to the network edge. This paper addresses a cooperative caching scenario in F-RAN, where each mobile user can acquire the requested contents from any one of its associated fog-computing-based access points (F-APs). However, caching disparate contents in different F-APs will lead to different content delivery delays, since mobile users suffer from diverse channel fadings and interferences when they download contents from different F-APs. Considering limited caching storage in each F-AP, diverse user preferences, unpredictable user mobility and time-varying channel states, an average transmission delay minimization problem is formulated. With the aid of dueling deep-Q-network framework, a delay-aware cache update policy is proposed for mobile users in F-RAN. The proposed cache update policy will decide to replace the stored contents in F-APs with the proper contents at each time slot. Compared with first in first out, least recently used and least frequently used caching policies, simulation experiments are performed to evaluate the performance of the proposed algorithm. Simulation results illustrate that the proposed caching policy yields better average hit ratio and lower average transmission delay than other traditional caching policies.