Dynamic Resource Management for Federated Edge Learning With Imperfect CSI: A Deep Reinforcement Learning Approach

Federated edge learning (FEL) has become a research hotspot to relieve the computational burden on servers and protect users’ data privacy. In an FEL system, adjusting the client selection and resource allocation scheme can reduce the energy consumption and improve the learning accuracy. However, obtaining a high learning accuracy and low energy consumption are primary challenges for FEL when the channel state information (CSI) is imperfect and the resources are dynamic. With this concern, to balance the learning accuracy and energy consumption, we formulate a joint client selection and dynamic resource allocation problem for FEL with imperfect CSI. The optimization problem is formulated as a markov decision process (MDP) that defines the state space, action space and reward function. To cope with traditional optimization algorithms’ inefficiency in solving the formulated problem, a deep reinforcement learning (DRL)-assisted method is used. We use a softmax deep double deterministic policy gradient (SD3) framework to train the model. Furthermore, a novel SD3-based FEL algorithm (FL-SD3) is proposed for client selection and dynamic resource allocation. Simulation results show that the proposed FL-SD3 improves the success rate by 11.6%, whilst the accuracy-to-energy (AE) gain is improved by 70.1% compared with some existing methods.