Optimal Minimization of the Covariance Loss

Let
$X$
be a random vector valued in
$\mathbb {R}^{m}$
such that
$\|X\|_{2} \le 1$
almost surely. For every
$k\ge 3$
, we show that there exists a sigma algebra
$\mathcal {F}$
generated by a partition of
$\mathbb {R}^{m}$
into
$k$
sets such that
$\|\mathrm {Cov}(X) - \mathrm {Cov}(\mathbb {E}[X\mid \mathcal {F}]) \|_{\mathrm {F}} \lesssim \frac {1}{\sqrt {\log {k}}}$
. This is optimal up to the implicit constant and improves on a previous bound due to Boedihardjo, Strohmer, and Vershynin. Our proof provides an efficient algorithm for constructing
$\mathcal {F}$
and leads to improved accuracy guarantees for
$k$
-anonymous or differentially private synthetic data. We also establish a connection between the above problem of minimizing the covariance loss and the pinning lemma from statistical physics, providing an alternate (and much simpler) algorithmic proof in the important case when
$X \in \{\pm 1\}^{m}/\sqrt {m}$
almost surely.