Enhancing the Generalization of Personalized Federated Learning with Multi-head Model and Ensemble Voting

Federated Learning has emerged as a transformative paradigm in the realm of collaborative machine learning, enabling the training of global models across decentralized devices without the need for centralizing data. While Federated Learning has shown remarkable promise, a critical limitation lies in its ability to personalize models to individual clients. Current approaches predominantly emphasize improving the accuracy of trained clients, inadvertently sidelining the significance of accommodating unseen clients. Furthermore, most of the existing personalized federated learning approaches require new clients to provide labeled data and undergo extensive retraining, posing a substantial barrier and hindering the broader adoption and engagement of potential users within these systems.In this paper, we introduce a novel and comprehensive solution to address these challenges: a generalized method for Personalized Federated Learning. Our approach transcends the limitations of conventional Federated Learning techniques by not only optimizing the accuracy of trained clients but also ensuring exceptional performance among unseen clients, even in diverse settings. Throughout extensive experiments, our method demonstrates significant improvements concerning the performance of seen and unseen clients, respectively, while eliminating the need for labeled data and model re-training among unseen clients.