PPNNBP: A Third Party Privacy-Preserving Neural Network With Back-Propagation Learning

With the advances in machine learning techniques and the potency of cloud computing there is an increasing adoption of third party cloud services for outsourcing training and prediction of machine learning models. Although cloud-hosted machine learning services enable more efficient storage and computation of data, privacy concerns and data sovereignty issues remain a major challenge. Privacy-preserving machine learning provides a promising solution. In this paper, a privacy-preserving neural network generation and utilization framework is presented, the PPNNBP framework. PPNNBP allows model training and prediction to be securely delegated to a third party with minimal data owner participation once the input data have been encrypted without recourse to secret sharing or multiple party setting. This is achieved using a proposed fully homomorphic encryption scheme, the Modified Liu Scheme (MLS), that permits certain operations over cyphertexts and features order preservation. The PPNNBP framework using MLS addresses the challenge of computational complexity of model learning using existing schemes; a complexity caused by the increasing size of cyphertexts (cyphertext inflation) and the quantity of noise introduced into cyphertexts through the application of multiplication operations, as learning progresses. Both the PPNNBP framework and MLS are fully described and analysed. The reported evaluation demonstrates that the PPNNBP framework achieves accuracy that is comparable to that obtained using a “standard” framework, whilst at the same time operating in a secure manner with minimal data owner participation.