Securing Fast Learning! Ridge Regression over Encrypted Big Data

Ridge regression is an important algorithm in machine learning and has been widely used in real-world applications like recommendation systems. Trained with a large training dataset, it outputs a curve that models the relationship between a scalar dependent variable and one or more explanatory variables. In general, the more training dataset it is fed, the more accurate the resulting model will be. In this era of "Big Data", however, data is usually split among different users. Exposing data to the other parties arouse privacy violation. It is difficult to apply conventional algorithms when the contributing users care about their privacy. In this work, we propose a new efficient privacy-preserving ridge regression scheme. We first design a packed secure multiplication protocol by utilizing Paillier encryption that is applicable to real numbers. Then we transform the ridge regression problem into the problem of solving linear equations, so we can employ Gaussian elimination and Jacobi iterative method to efficiently derive the learned model. Finally, extensive experiments on real-world datasets are conducted to show that our scheme outperforms the state-of-the-art solutions in terms of both computation and communication efficiencies, and incurs negligible errors compared with performing ridge regression in the clear.