SAM: An Efficient Approach With Selective Aggregation of Models in Federated Learning

Federated learning (FL) is a promising distributed learning mechanism that revolutionizes our interaction with data in the IoT ecosystem. Due to the rapidly growing scale of smart devices and the limited transmission resources of networks, a simple, consistent, and scalable FL framework aiming to address the communication bottleneck is urgently needed. In this work, we propose an efficient approach with selective aggregation of models (SAMs) to mitigate the communication overload in FL systems. The introduction of SAM enables each local client to upload its model with a certain probability, resulting in a significant reduction in costly communication expenses. We design the algorithm for SAM, analyze the convergence bound on nonconvex objectives for heterogeneous data, which illustrates the impact of the selection probability as well as the set size of participating clients on the system performance, and assess the conservation for the network resource utilization by modeling queuing systems. We conduct various experiments to evaluate the performance of SAM, whose outcomes suggest that significant alleviation of the communication bottleneck can be accomplished with marginal cost of performance loss. It will also be shown that SAM is a communication-efficient method that can be freely applied to other frameworks.