Scalable Differentially Private Model Publishing Via Private Iterative Sample Selection

Model publishing and deployment are essential for artificial intelligence applications. A major challenge in model publishing is efficiently distributing the models in a scalable way without violating the privacy of sensitive data. With the wide adoption of machine learning techniques, the privacy concern has also drawn much attraction. Differential privacy has become an important notion for privacy protection and is popular in private learning. However, it may bring much accuracy loss to fulfill data privacy. In addition, the private models are also hard to train in terms of convergence, which makes the existing approaches not scalable for private model publishing. This paper proposes a model publishing framework that provides a novel way to train privacy-preserving machine learning models with fast convergence and a lower privacy budget. By incorporating the concept of iterative machine teaching and the techniques in differential privacy, we have explored a way to privately select more suitable examples in the training process for achieving good accuracy with fewer iterations. Our analysis shows the privacy and convergence performance of the proposed method, and extensive experiments have been performed on real-world datasets to demonstrate its effectiveness.