CFPA: Cognitive Federated Partial Adaptation for Effective Personalization

Federated Learning (FL) is a promising machine learning paradigm to train a global model from multiple clients while ensuring privacy. A key challenge in FL is the data heterogeneity that impairs the generalization of the single global model on each client. As a prevalent solution, the existing personalized FL primarily concentrates on the client-level personalization process, which may pose an overfitting problem since the local data of a single device is usually limited. Additionally, these methods lack the cognitive ability for collaborative learning among similar clients, which also restricts the performance of personalized models. To address this, we propose CFPA, a novel communication-efficient, and computation-efficient Cognitive Federated Partial Adaptation framework to boost personalization performance via the Soft-grouping Weighted Aggregation (SWA) strategy. Specifically, CFPA decouples the model into the body and the head and requires clients collaboratively train a well-performing federated pre-personalized model. Then, CFPA leverages the distribution knowledge extracted from the output feature maps of the convolutional layers in the federated pre-personalized model to identify the similarity among clients efficiently. Guided by the similarity matrix, CFPA further performs weighted federated adaption on the head of each local model, ultimately generating a personalized local model for each client. Comprehensive experiments on three benchmark datasets with various heterogeneous settings demonstrate that CFPA outperforms other state-of-the-art FL approaches.