A parallel implementation of Singular Value Decomposition based on Map-Reduce and PARPACK

In the e-commerce on the Web, recommender systems become a powerful technology for extracting valuable information from its customer databases. These systems also help customers find products they want to buy from a business sites. Singular Value Decomposition(SVD) is a useful technology to speedup the recommendations with very fast online performance, requiring just a few simple arithmetic operations. Unfortunately, computing the SVD of a large scale matrix is very expensive. In this paper, we propose to parallelize the SVD algorithm to run on distributed computers. Our parallel algorithm employs a parallel ARPACK algorithm to perform parallel eigenvalue decomposition. Experimental results show that the proposed method can significantly speed up the SVD computation cost while providing comparable prediction quality.