Personalized Federated Learning via Gradient-Fusion and Gradient-Decoupling for Heterogeneous Data

Federated Learning (FL) enables devices to collaboratively train a global model without centralizing raw data. However, heterogeneous data distributions often hinder a single global model from performing optimally across diverse clients. In this paper, we introduce two personalized federated learning methods to address the aforementioned challenge. First, we introduce a gradient-fusion approach that merges global gradients aggregated via a learnable collaboration matrix with local gradients tailored to each client's data. This fusion harmonizes collective intelligence with device-specific knowledge, thereby improving personalization under data heterogeneity. Second, we propose a gradient-decoupling method that mitigates the limitations of relying solely on a collaboration matrix for balancing knowledge sharing and personalization. By decoupling the tasks and regularizing local updates through the incorporation of extracted collective intelligence, we introduce an inductive bias that significantly improves generalization. Extensive evaluations on four computed tomography and pathology imaging datasets covering a wide range of heterogeneity show that our frameworks substantially outperform existing baselines. We also provide theoretical analyses that establish performance bounds and convergence guarantees under mild assumptions.