Joint Client Selection and Privacy Compensation for Differentially Private Federated Learning

Differentially private federated learning learns a global machine learning model from clients' distributed private data while providing privacy protections. However, clients under differential privacy protection still sustain a certain degree of potential privacy leakage, which can result in the clients' reluctance to participate in model training. Hence, both client selection and privacy compensation are important decisions to determine which clients will join the learning. In particular, since a client's privacy leakage increases with his participation frequency, the client selection decision is tightly coupled with the privacy compensation. This paper proposes a joint client selection and privacy compensation Bayesian-optimal mechanism design approach. Despite being a challenging non-convex optimization problem, we propose an efficient algorithm to solve it. We first characterize the optimal selection probability of clients with heterogeneous privacy sensitivities. This characterization significantly reduces the dimension of the problem and allows us to propose an efficient algorithm to solve the problem. Numerical results show that the proposed mechanism Pareto dominates the unbiased selection-based mechanism in terms of both test accuracy and monetary costs. Specifically, our mechanism decreases the test loss by up to 81.7% under the same monetary cost.