Exploring the Architecture of Multiple GEMM Accelerators in Heterogeneous Systems

General Matrix-Matrix Multiplication (GEMM) is a commonly used kernel in machine learning, scientific computing and many other applications. Designing a customized GEMM accelerator can bring obvious performance and power consumption benefits. In this paper, we first perform a detailed workload characterization for different sizes of GEMM kernels. Then, we make a comprehensive design space exploration to find the Pareto optimal architecture configurations. Lastly, we compare two versions of multiple GEMM accelerator systems with main-stream BLAS libraries (e.g., OpenBLAS, MKL and cuBLAS). The proposed GEMM acceleration hardware system shows higher energy efficiency than existing software implementations.