Sequential Privacy Budget Recycling for Federated Vector Mean Estimation: A Game-Theoretic Approach

Privacy-preserving vector mean estimation is a crucial primitive in federated analytics. Existing practices usually resort to Local Differentiated Privacy (LDP) mechanisms that inject random noise into users’ vectors when communicating with users and the central server. Due to the privacy-utility trade-off, the privacy budget has been widely recognized as the bottleneck resource that requires well-provisioning. In this paper, we explore the possibility of privacy budget recycling and propose a novel ChainDP framework enabling users to carry out data aggregation sequentially to recycle the privacy budget. We establish a sequential game to model the user interactions in our framework. We theoretically show the mathematical nature of the sequential game, solve its Nash Equilibrium, and design an incentive mechanism with provable economic properties. To alleviate potential privacy collusion attacks, we further derive a differentially privacy-guaranteed protocol to avoid holistic exposure. Our numerical simulation validates the effectiveness of ChainDP, showing that it can significantly save privacy budget as well as lower estimation error compared to the traditional LDP mechanism.

I. Introduction

Vector mean estimation is a key operation and basic building block in many applications for federated analytics, e.g., federated learning [2] and frequency estimation [3]. In federated learning, in each training round, each user trains machine learning models locally and then uploads the trained parameters vector to the server. The server aggregates the received parameters by vector mean estimation. Frequency estimation can also be regarded as a special case of vector mean estimation where each user owns a binary vector indicating whether the user owns each of the items in some universe, and the server wants to estimate the frequency of each item. Many of these applications are being widely deployed by companies such as Apple [4], Google [5], and Microsoft [6].