Adaptive Federated Learning via Mean Field Approach

In order to solve the problem of "data island" and preserve individual’s privacy, federated learning, as a distributed machine learning technology, has emerged recently. In federated learning, the model training is distributed over edge clients and coordinated by a central server. Each client only needs to send the updated model parameter to the central server for aggregation without sharing its private data. However, due to the data divergence of heterogeneous clients, the convergence rate of the global model training may be very slow, especially for non-IID data case. To deal with this issue and achieve fast convergence, we propose an adaptive learning rate strategy for each client by considering the deviation of the local model parameter from the global model parameter at each global training iteration. To enable decentralized learning rate design for each client, a mean-field scheme is introduced to estimate the global model parameters over time, which does not even require many clients to communicate frequently. Finally, we run numerical experiments to validate our results.