Energy-Efficient DNN Inferencing on ReRAM-Based PIM Accelerators Using Heterogeneous Operation Units

Operation Unit (OU)-based configurations enable the design of energy-efficient and reliable ReRAM crossbar-based Processing-In-Memory (PIM) architectures for Deep Neural Network (DNN) inferencing. To exploit sparsity and tackle crossbar non-idealities, matrix-vector-multiplication (MVM) operations are computed at a much smaller level of granularity than a full crossbar, referred to as OUs. However, determining the suitable OU size for a given DNN workload presents a non-trivial challenge as the DNN layers exhibit different levels of sparsity and have varying impact on overall predictive accuracy. In this paper, we propose a framework for designing heterogeneous OU-based PIM accelerators. The OU configurations vary based on the characteristics of the neural layers and the time-dependent conductance drift of PIM devices due to repeated inference runs. Overall, our experimental results demonstrate that the sparsity-aware layer-wise heterogeneous OU-based PIM computation reduces latency and energy by 34% and 73% on average respectively, compared to state-of-the-art homogeneous OU-based architectures without compromising the predictive accuracy.