FedOES: An Efficient Federated Learning Approach

Federated Learning (FL) is a distributed approach for performing machine learning tasks. It prevents data sharing by aggregating the models trained by distributed clients on the central server, thereby maintaining data privacy. However, a major challenge in FL architecture is the high communication costs incurred by clients in multiple communication rounds, which can create communication bottlenecks. Additionally, the issue of non-independent and identical distribution (Non-IID) data poses a significant challenge. To address these challenges, our paper proposes a method based on in-cluster training and top-k gradient sparsification. The clients adopt the in-cluster training strategy to mitigate the negative effects of Non-IID data, while sparse gradient is implemented on both the clients and the server to reduce communication costs. Our experiments demonstrate that our proposed approach, FedOES, outperforms federated averaging in common FL scenarios. It not only improves communication efficiency, but also achieves excellent performance in scenarios where the data is Non-IID.