Auto-Tuning Strategies for Parallelizing Sparse Matrix-Vector (SpMV) Multiplication on Multi- and Many-Core Processors

Because sparse matrix-vector multiplication (SpMV) is an important and widely used computational kernel in many real-world applications, it behooves us to accelerate SpMV on modern multi- and many-core architectures. While many storage formats have been developed to facilitate SpMV operations, the compressed sparse row (CSR) format is still the most popular and general storage format. However, parallelizing CSR-based SpMV on multi- and many-core processors (e.g., CPUs, APUs, GPUs) remains a challenging problem, including dealing with uncoalesced memory access, balancing workload, and identifying the most appropriate parallelizing strategy. In the paper, we propose a novel auto-tuning framework that automatically finds the most efficient parallelizing strategy to achieve high-performance SpMV. Our framework can deter- mine the right binning schemes to group similar workloads into bins (e.g., buckets) with negligible overhead. Then, for each bin, the most suitable kernel is selected to process the rows within. Our framework is input-aware and based on a machine-learning method. The results show that our auto-tuned SpMV performs significantly better than the default SpMV. The speedups on 16 representative matrices range from 1.2x to 52.0x. Compared to the state-of-the-art SpMV kernel, our work yields better performance in most cases, achieving up to a 1.9x speedup.