Federated Deep Reinforcement Learning for Delay and Energy Consumption Tradeoff in Scalable Cell-free Mobile Edge Computing Networks

The allocation of communication and computational resources in cell-free mobile edge computing (CF-MEC) networks is investigated in this study. Under the constraints of maximum transmission delay, offloading rate of user equipment (UE), and power control factors, a CF network for urban scenarios is established and an optimization problem aimed at minimizing the weighted sum of delay and energy consumption is proposed. Due to the frequent signaling interactions between the access point (AP) and central processing unit (CPU) of CF networks, as well as the complexity and non-convexity of the proposed problem, an algorithm of scalable cell-free federated deep deterministic policy gradient (CFF-DDPG) has been proposed to solve the original problem. Firstly, dynamic clustering of all APs is implemented to enhance the scalability of the CF network. Secondly, a federated approach is adopted to adapt to the CF network, aiming to reduce the complexity of signaling interactions between APs and the CPU. Specifically, the deep deterministic policy gradient (DDPG) algorithm is utilized for local training at the UE, followed by federated averaging at the CPU of the CF network. Simulation results show that the proposed CFF-DDPG algorithm provides significant performance improvement over the three baseline algorithms, in addition to reducing the total cost by 78% compared to small-cell MEC, and can effectively enhance the scalability of traditional cell free networks.