Batched SVD on CPU-GPU based on integer programming

Singular value decomposition (SVD) is a commonly employed matrix factorization. In real-world applications, the data requiring SVD is usually batched in small matrices within a size not greater than 1024 times 1024. However, the professional high performance libraries on CPU like MKL do not provide any interfaces for batched SVD while the libraries on GPU such as Nvidia cuSOLVER do not provide ones within matrices greater than 32 times 32. Although several studies aim to address this problem, they only focus on GPU’s computing power but ignore CPUs’. In this paper, we improve the batched SVD from the novel perspective of heterogeneous computing within matrices greater than 32 times 32 but not greater than 1024 times 1024. We propose a methodology based on statistics and integer programming to partition data between CPU and GPU. The evaluation demonstrates that compared to cuSOLVER, our proposed methodology can achieve a speedup of 1.4x - 1.9x.