Automatic Thread-Block Size Adjustment for Memory-Bound BLAS Kernels on GPUs

The performance of a CUDA kernel often depends on the number of threads per thread-block (thread-block size), and the optimal configuration differs according to the graphics processing unit (GPU) hardware and the given data size to the kernel. In particular, in linear algebra libraries such as Basic Linear Algebra Subprograms (BLAS), most routines support a wide range of problem sizes and various processors with different architectures or number of cores. Therefore, we need a method to adjust the thread-block size automatically in an economical and theoretical manner as much as possible depending on the circumstances of the routine call. In this study, we propose a method to adjust the thread-block size for several memory-bound BLAS kernels on NVIDIA GPUs. Our method is a model-driven approach that can automatically determine the thread-block size on the basis of three occupancy models on the warp, thread-block, and grid level before every launch of the kernel. We demonstrate that our method determines nearly optimal thread-block size for several kernels on Kepler and Maxwell architecture GPUs.