Novel Deep Reinforcement Learning for User Association in Fog Radio Access Networks

As an evolution of cloud radio access network (C-RAN), fog radio access network (F-RAN) becomes promising for future mobile communications by enabling processing and caching at fog access points (FAPs). Different from the centralised C-RAN, F-RAN has a semi-distributed architecture, aiming to alleviate traffic load on the fronthaul links in C-RAN. Under the semi-distributed architecture in F-RAN, which employs a cell-free multiple input multiple output (MIMO) access technique, decisions on the joint user-FAP association and transmit power allocation are made at individual FAPs. To mitigate strong interference, FAPs will need to exchange cooperative status information, such as CSI, user association details or transmission power levels. However, this can lead to significant communication overhead within the network and introduce high complexity in the decision-making process. In this paper, accounting for the semi-distributed nature of the F-RAN architecture, reinforcement learning is leveraged as a potential solution to this kind of problem, and a novel multi-agent dual deep Q-network (MA-DDQN) algorithm is proposed by introducing experience exchange in partially observable Markov decision process environments. The simulation results show that the proposed reinforcement learning based algorithm outperforms the DDQN algorithm as well as the existing low-complexity algorithms.