Information-Exchangeable Hierarchical Clustering for Federated Learning With Non-IID Data

Federated Learning (FL) allows Internet-of-Things (IoT) devices to train a global model collaboratively and keep their data locally to address privacy concerns. However, the current FL framework has three main drawbacks, high communication cost, single point of failure, and low accuracy on non-independent and identically distributed (non-IID) data. To this end, we propose a novel FL framework, IHC-FL, to 1) group devices into clusters based on communication cost and model distance, 2) distribute model aggregation over cluster heads, and 3) construct a topology to guide cluster heads to exchange model updates. To the best of our knowledge, this paper makes the first attempt to jointly optimize grouping user devices into clusters and exchanging model updates among cluster heads to enhance model performance. The numeric results show that IHC-FL can reduce 38%~89% of total communication cost over time than other heuristics with non-IID data on FMNIST and CIFAR-10 to achieve the target accuracy.