A PPO-Based Dynamic Asynchronous Semi-Decentralized Federated Edge Learning

Federated edge learning (FEEL) is gaining increasing attention due to its characteristics of privacy protection, low latency, and low communication overhead. However, it still faces challenges such as a single point of failure and the imbalance between communication efficiency and training efficiency caused by heterogeneous clients. To address the above issues, we investigate a semi-decentralized FEEL (SD-FEEL) architecture, where edge servers train their local models with the associated clients in a centralized manner and exchange models with their one-hop neighbors distributively. For the upper-layer edge servers, a fully asynchronous aggregation mechanism is proposed to accelerate the model diffusion, while for the lower-layer clients, the proximal policy optimization (PPO) algorithm is employed to dynamically select the aggregation time based on currently available resources. Simulation results show that the proposed algorithm can effectively balance training efficiency and communication efficiency. Besides, the model accuracy of the proposed algorithm is improved by around 4.5% and 11% compared to asynchronous FL and synchronous FL, respectively.