SnapCFL: A Pre-Clustering-Based Clustered Federated Learning Framework for Data and System Heterogeneities

Federated Learning (FL) has emerged as a promising framework to address data privacy concerns associated with mobile devices, in contrast to conventional Machine Learning (ML). However, traditional FL encounters significant challenges due to the heterogeneities among different clients. Clustered Federated Learning (CFL) has demonstrated effectiveness in mitigating the data heterogeneity challenge, which significantly limits a broader application of FL. Nevertheless, existing CFL approaches often tightly couple the clustering process with the main FL process, affecting the flexibility and performance of CFL. In this paper, we propose a pre-clustering-based CFL approach, named SnapCFL, which decouples the CFL process into pre-clustering and main FL stages, considering both the impact of heterogeneity on CFL accuracy and the framework's flexibility. The pre-clustering stage models the measurement of data similarity as a two-sample hypothesis testing problem to more accurately group clients and alleviate data heterogeneity. In the main FL stage, a constraint-based client selection method is employed to address the system heterogeneity problem. We conduct extensive experiments using popular datasets with various heterogeneity settings. The results demonstrate that SnapCFL achieves excellent performance in terms of accuracy and efficiency. Compared to five other state-of-the-art approaches, SnapCFL can improve model accuracy by 0.7%$\sim$36.4%, and achieve the same level of accuracy with at least 0.08× the convergence time.