Novel Speedup Techniques for Parallel Singular Value Decomposition

Singular Value Decomposition (SVD) is a fundamental problem in linear algebra and it has applications in many fields. For example, as a data reduction technique, SVD is widely used for Principal Component Analysis (PCA). It also has extensive applications in the machine learning domain. For example, it is widely used in feature selection, data clustering, classification, and regression. Traditional sequential SVD algorithms take cubic time which may not be affordable especially when the input matrix is large. Therefore, there is a pressing need to develop efficient sequential and parallel SVD algorithms. Jacobi iteration, as one of the most common SVD algorithms, has attracted a lot of attention. Among many algorithms that have been proposed, there are some relaxation schemes that have been proven to be very effective in parallel Jacobi-SVD computing. Such relaxation schemes are platform and architecture independent. In this paper, we propose some novel techniques to further generalize and speedup these relaxation schemes. Theoretical analysis proves the efficiency of our algorithms. Experimental results also show that our algorithms outperform the existing ones.