FedHKD-SA: Bilateral optimization method for combating forgetting in personalized federated learning

Federated learning (FL) is an emerging paradigm in edge computing that facilitates the training of machine learning models across numerous resource-constrained edge devices without the necessity of transferring data to a centralized server. A significant challenge in federated learning is the statistical heterogeneity of the client data distribution, which can severely undermine the generalizability and performance of the models in individual clients, particularly in the context of the Internet of Things (IoT). In this paper, we observe that the initialization process in each communication round results in the loss of personalized knowledge on the clients, which is closely related to the above challenges. Based on this observation, we propose a novel personalized federated learning framework, FedHKD-SA, which enables clients to store and refine the knowledge from historical knowledge models in each round of local model training, accelerating the recall of personalized knowledge for the most recently initialized client models. Moreover, the second aggregation on the server side facilitates targeted learning of the localized personalized knowledge conveyed by local models, skillfully maintaining a balance between generalization and personalization, and accelerating model convergence. Extensive experiments across various datasets and settings demonstrate the effectiveness and robustness of our framework.