AutoRelax: HW-SW Co-Optimization for Efficient SpGEMM Operations With Automated Relaxation in Deep Learning

We propose a HW-SW co-optimization technique to perform energy-efficient spGEMM operations for deep learning. First, we present an automated pruning algorithm, named AutoRelax, that allows some level of relaxation to achieve higher compression ratio. As the benefit of the proposed pruning algorithm may be limited by the sparsity level of a given weight matrix, we present additional steps to further improve its efficiency. Along with the software approach, we also present a hardware architecture for processing sparse GEMM operations to maximize the benefit of the proposed pruning algorithm and sparse matrix format. To validate the efficiency of our co-optimization methodology, we evaluated the proposed method on three benchmarks, language modeling, speech recognition and image classification. As a result, our approach improves the on-chip performance on spGEMM operations by 9.50
$\sim$
27.57% and achieves 15.35
$\sim$
33.28% energy reduction considering DRAM accesses over other sparse accelerators.