Towards Efficient Federated Learning Framework via Selective Aggregation of Models

Federated Learning (FL) is an important distributed learning paradigm that focus on data privacy and system efficiency. With the rapid growth of the edge devices and the limited resources of the network, a simple, consistent and scalable FL framework aiming at alleviating the communication bottleneck is urgently needed. In this work, we propose SAM, an efficient FL mechanism with Selective Aggregation of Models, which allows each local client to upload their models with a certain probability. We develop the algorithm for SAM, derive the convergence bound on non-convex objectives for heterogeneous data distribution, and conduct several experiments to evaluate the performance, as well as the effective utility range of SAM. The numerical results corroborate the theoretical analysis, and demonstrate a considerable reduction in communication overloads with marginal cost of performance loss when the selective probability is within the effective utility range. As a model selection mechanism, SAM has great potential when integrated with other federated framework to collaboratively optimize the communication workloads.