Privacy-Preserving Distributed Deep Learning with Privacy Transformations

Distributed Deep Learning (DDL) allows disparate sites or entities to use their local data to collaboratively learn a model at a central server. To protect data privacy, existing approaches like fully homomorphic encryption and differential privacy are either computationally prohibitive or insecure. In this paper, we proposed applying a privacy-preserving transformation (PPT) before sending the transformed data to the server. The design goals of PPT include computation efficiency, privacy preservation, and good learnability at the server with maximal reuse of DL software infrastructure. After analyzing the security model and possible attacks, we evaluated simple PPTs including scrambling, random linear transforms, and Advanced Encryption Standard (AES). While AES is more secure than the others, it significantly degrades the learning performance. To address this challenge, we proposed a novel random deep neural network as PPT. Our experiments showed that the random weights and connections provide adequate security and good learning performances at the server.