Learning with User-Level Differential Privacy Under Fixed Compute Budgets

We investigate practical and scalable algorithms for training machine learning models with user-level differential privacy (DP) in order to provably safeguard all the examples contributed by each user. Motivated by the application of large language model (LLM) fine-tuning, we analyze algorithms under fixed compute budgets, especially large budget settings. We study two variants of DP-SGD with: (1) example-level sampling (ELS) and per-example gradient clipping, and (2) user-level sampling (ULS) and per-user gradient clipping. We derive a novel user-level DP accountant that allows us to compute provably tight privacy guarantees for ELS. We show that for fixed compute and privacy budgets, ULS generally yields better results than ELS, especially when each user has a diverse collection of examples and the compute budget is large. We validate our findings through experiments in synthetic mean estimation and LLM fine-tuning tasks under fixed compute budgets. We find that ULS is significantly better in settings where either (1) strong privacy guarantees are required, or (2) the compute budget is large. Notably, our focus on scalability enables us to scale to models with hundreds of millions of parameters and datasets with hundreds of thousands of users.