Accelerating Sparse Matrix-Matrix Multiplication by Adaptive Batching Strategy on MGPUSim

Sparse Matrix-Matrix Multiplication (SpMM) is a widely used algorithm in Machine Learning, particularly in the increasingly popular Graph Neural Networks (GNNs). SpMM is an essential arithmetic operation in GNNs and has been parallelized on various platforms to accelerate GNN training. However, it has not been deeply studied on multi-GPU architectures.In this work, we parallelize the SpMM algorithm on MGPUSim, including the parallel execution mechanism and architecture design. More importantly, we propose an adaptive batching strategy (ABS) to handle the irregular memory access of sparse matrices and allocate work-item resources efficiently. The strategy addresses issues such as scheduling overhead caused by too large work-groups, resource contention, and poor parallelism resulting from performance degradation due to too small work-groups. ABS improves the GPU’s command processor scheduling speed, increases the efficiency of the incoming request rate, and optimizes work-group overhead. Finally, we conducted experiments on the i9-10900F CPU and NVIDIA RTX 3070 GPU using a set of matrices from SuiteSparse. Experimental results show that ABS achieves an average performance acceleration of 1.53× compared to the baseline and 1.40× compared to CUDA. Compared to the latest SpCaches approach used on FPGAs, our approach achieves a 1.12× relative performance improvement.