Towards an Efficient Federated Learning Framework with Selective Aggregation

Federated Learning shows promise for collaborative, decentralized machine learning but faces efficiency challenges, primarily network straggler-induced latency bottlenecks and the need for complex aggregation techniques. To address these issues, ongoing research explores asynchronous FL, i.e., federated learning models, including an Asynchronous Parallel Federated Learning [5] framework. This study investigates the impact of varying worker node numbers on key metrics. More nodes offer faster convergence but may increase communication overhead and straggler vulnerability. We aim to quantify how the number of worker node variations for one global aggregation can affect convergence speed, communication efficiency, model accuracy, and system robustness, optimizing asynchronous FL system configurations. This work is crucial for practical and scalable FL applications, mitigating network stragglers, data distribution, and security challenges. This work analyses Asynchronous Parallel Federated Learning and showcases a paradigm shift in the approach by selectively aggregating early arriving worker node updates with a novel parameter ‘x’, improving efficiency and reshaping FL.

I. Introduction

Federated Learning (FL) is an innovative approach to machine learning that allows multiple devices to collaboratively train a shared model while preserving the privacy of their data. This decoupling of model training from direct data access is a game-changer, particularly in sensitive fields like biomedicine and finance, where data privacy and security are paramount. Most existing FL methods, including the pioneering FedAvg [3], operate synchronously (SyncFL), involving a central server broadcasting the global model, edge devices updating their local models with private data [1], [2], [4], and the central server aggregating updates to create the next global model. However, in real-world scenarios marked by device heterogeneity, two challenges hinder SyncFL's efficiency: