Accelerating Look-Up Table based Matrix Multiplication on GPUs

Multiplying matrices is among the most fundamental and compute-intensive operations in machine learning. Approximated Matrix Multiplication (AMM) based on table look-ups can significantly reduce the pressure on computing units and memory bandwidth, and has great potential in large-scale machine learning applications. In this work, we speed up table look-ups on GPUs to improve the performance of matrix multiplication. To avoid random memory accesses in table look-ups, we propose a novel warp-wide data sharing execution model. With this execution model, we develop a GPU AMM library to speed up MADDNESS (the state-of-the-art AMM), named GPU-MADDNESS. The experimental results show that GPU-MADDNESS improves the performance by 103X on average, and outperforms the tiling implementation by up to 42%.