RAC-NAF: A Reconfigurable Analog Circuitry for Nonlinear Activation Function Computation in Computing-in-Memory

The emerging computing-in-memory (CIM) architecture shows promise in efficiently processing deep neural networks (DNNs) by minimizing the data movement through analog in-memory computing. Unfortunately, the energy efficiency of CIM is still limited since it struggles to efficiently process the massive nonlinear activation functions (AFs) widely used in DNNs. In the current solutions, AFs require to be fulfilled in the digital domain with co-processors or lookup table (LUT). Therefore, a significant amount of data requires a round-trip conversion between the analog and digital domains at each AF process. The unavoidable analog-to-digital and digital-to-analog (AD/DA) conversions dominate the system’s power consumption and reduce the overall energy efficiency. To address these issues, we propose a reconfigurable analog circuitry for nonlinear AF computation, named RAC-NAF. It is a pure analog circuitry that utilizes Taylor approximation (TA) to fit the arbitrary AFs, thereby reducing AD/DA conversions. To enhance the accuracy, RAC-NAF adopts a segmentation calculation method (SCM) based on the characteristics of nonlinear AFs. Moreover, the RAC-NAF provides the capability for reconfiguration to support various AFs and can be easily integrated with the existing CIM accelerators. The experimental results show that the proposed RAC-NAF significantly improves the performance of the CIM accelerators and reduces the energy consumption. When performing inference on various CIM accelerators, the energy consumption of AD/DA conversions can be reduced up to $12.31{\times }$ , while the overall energy efficiency can be increased by $2.34{\times }$ – $5.20{\times }$ , and the accuracy loss is below 1%.