On the Gibbs Exponential Mechanism and Private Synthetic Data Generation

We study the Gibbs exponential mechanism and its use in generating private synthetic data. We first present bounds on the expected utility of the Gibbs and generalized Gibbs mechanisms based on their privacy parameters. Next, we provide two notions of privacy and functionals of utility with respect to which the Gibbs mechanism is provably optimal among all mechanisms. These rely on a valuable property of Gibbs distributions relating them to relative entropy, called the Gibbs variational principle, and its extension to Rényi divergences and generalized moments. Finally, we study how to use the Gibbs mechanism to generate synthetic data privately. Combining known results in empirical process theory with the privacy-utility tradeoff results of this paper, we derive bounds on the utility of the Gibbs mechanism as a function of the size of the synthetic database.