PSO-DPSGD: An Improved Method for Differentially Private Deep Learning Based on Particle Swarm Optimization

Existing approaches for differentially private deep learning with record-level differential privacy (e.g., DPSGD) involve bounding the contribution of each data point to the gradient by clipping it to an empirical threshold. However, determining an optimal clipping norm is challenging, as the gradient norm distribution varies significantly depending on factors such as the model architecture and training data. We propose a method that leverages swarm optimization techniques, Particle Swarm Optimization (PSO), to determine the clipping norm. The proposed method formulates the search for the optimal norm as an intelligent optimization problem, enabling integration within the DPSGD framework. Theoretical analysis suggests that PSO-optimized clipping improves the privacy-utility trade-off compared to the empirical norm. Future work will explore its performance across various deep-learning tasks and validate its effectiveness through empirical experiments.