Privacy Preserving Inference with Convolutional Neural Network Ensemble

Machine Learning as a Service on cloud not only provides a solution to scale demanding workloads, but also allows broader accessibility for the utilization of trained deep neural networks. For example, in the medical field, cloud-based deep-learning assisted diagnoses can be life-saving, especially in developing areas where experienced doctors and domain expertise are lacking. However, preserving end-users' data privacy while using cloud service for deep learning is a challenge. Some recent works based on fully homomorphic encryption have enabled neural-network predictions on encrypted input data. In this paper, we further extend the capability of privacy preserving deep neural network inference, through a joint decision made by multiple deep neural network models on encrypted data, to address bias caused by unbalanced local training datasets. In particular, we design and implement a privacy preserving prediction method through an ensemble of convolutional neural networks. The extensive experiment results show that our method can achieve higher accuracy compared to individual models, and preserve the user data privacy at the same level. We also verify the time efficiency of our implementation.