Personalized Federated Learning With Multi-View Geometry Structure

Federated learning (FL) is a distributed machine learning paradigm ensuring data privacy. However, the statistical heterogeneity poses a challenge to building a single model that can perform well across all clients’ data distributions. Personalized federated learning (PFL) has emerged as a solution to mitigate the impact of statistical heterogeneity by training separate models for various target data distributions. A crucial aspect of PFL is to utilize additional information in the federated system to assist in training personalized models. In this paper, we introduce a novel PFL approach that leverages a multi-view geometry structure (GPFL). GPFL formulates an optimization problem to determine the correlation weights among clients by utilizing the geometry structure composed of client gradient updates. It further builds information carriers that facilitate personalized training based on the weights. To accurately capture the correlations, we use both L2 distance and cosine similarity views to depict geometric similarity. In federated training, the discrepancy in timeliness and tendency to overfit to local data in gradient updates cause the geometric similarity of such updates to inadequately reflect the client relationships. Therefore, GPFL employs representative gradients extracted from the client’s historical gradients to infer the correlation weights. Experimental results on four datasets, convex and non-convex objectives, and two federated learning settings demonstrate that our method outperforms several PFL methods.