Maintaining data mining accuracy on distorted datasets is an important issue in privacy preserving data mining. Using matrix approximation, we propose several efficient and flexible techniques to address this issue, and utilize unique characteristics of matrix factorization to maintain data pattern. We use the support vector machine classification to compare accuracy maintenance after data distortion by different methods. With better performance than some classical data perturbation approaches, nonnegative matrix factorization and singular value decomposition are considered to be promising techniques for privacy preserving data mining. Experimental results demonstrate that mining accuracy on the distorted data used these methods is almost as good as that on the original data, with added property of privacy preservation. It indicates that the matrix factorization-based data distortion schemes perturb only confidential attributes to meet privacy requirements while preserving general data pattern for knowledge extraction.