BafSP: Co-Design of Compute SRAM and Bit-Aware Data Flip Mitigation with In-Memory Sparsity Detection for SpMM

Sparse-dense matrix multiplication (SpMM) is a fundamental operator in Deep Neural Networks (DNNs). Previous works utilized Processing-In-Memory (PIM) architecture to execute SpMM, as it can reduce data movement. However, the uneven distribution of non-zeros in the sparse matrix and the tight data dependence between sparse and dense matrices pose challenges to execute SpMM on PIM efficiently. Furthermore, frequent data flipping and the presence of bit-level zeros in the elements further impede the efficiency of SpMM on PIM, and existing PIM frameworks exhibit limited flexibility in sparse processing for varying dimensions matrices. Therefore, we propose an algorithm, dataflow and SRAM-based PIM co-design to explore the optimization opportunity for SpMM. At the algorithmic level, this method mitigates the data flipping of bit-level granularity by implementing appropriate data conversions and compensations. At the dataflow level, a novel dataflow is adopted for flexible parallel detection of sparse matrices and efficient matrix calculation by coupling sparse detection with matrix calculation, while mitigating memory contention. At the circuit level, a novel digital SRAM PIM macro is proposed to detect bit-level sparsity in memory and support algorithm and dataflow. Experimental results show that the energy efficiency of the proposed design is 6.63-28.22 TOPS/W. Compared with the advanced works, this design achieves up to 3.46 x speedup and 6.43 x energy efficiency improvement.