Privacy-Preserving Constrained Quadratic Optimization With Fisher Information

Noisy (stochastic) gradient descent is used to develop privacy-preserving algorithms for solving constrained quadratic optimization problems. The variance of the error of an adversary's estimate of the parameters of the quadratic cost function based on iterates of the algorithm is related to the Fisher information of the noise using the Cramér-Rao bound. This motivates using the Fisher information as a measure of privacy. Noting that the performance degradation in noisy gradient descent is proportional to the variance of the noise, a measure of utility is defined to be equal to the variance of the noise. Trade-off between privacy and utility is balanced by minimizing the Fisher information subject to a constraint on the variance of the noise. The optimal privacy-preserving noise is proved to be Gaussian, which implies that the developed privacy-preserving optimization algorithm also guarantees differential privacy.