CFedKT: Clustered Federated Learning with Knowledge Transfer

Traditional federated learning (FL) aggregates the models from individual clients to obtain a globally more generalizable model. In practical scenarios, the effectiveness of aggregating a global model is compromised due to data heterogeneity. Clustered federated learning (CFL) partitions clients into clusters based on the correlations of their held data to mitigate the impact of data heterogeneity. However, the lack of global knowledge sharing limits the generalization ability of the cluster-based models. Therefore, we propose a novel cluster federated learning algorithm with knowledge transfer: CFedKT. The method uses a spectral clustering algorithm to cluster clients according to the weight update vectors of each client model and aggregates multiple cluster models to update the corresponding client model. We introduce a global model to facilitate knowledge transfer among models from different clusters, aiming to strike a balance between the performance of the global model and cluster-specific personalized features. In the experiments, CFedKT demonstrated significant improvements in accuracy and communication efficiency across different tasks and diverse data heterogeneity scenarios. Additionally, CFedKT showed a certain level of robustness and effectiveness under varying degrees of data heterogeneity.