LIORAT: NN Layer I/O Range Training for Area/Energy-Efficient Low-Bit A/D Conversion System Design in Error-Tolerant Computation-in-Memory

Analog Computation-in-Memory (CiM) with ReRAM accelerates the MAC operations of neural networks (NNs). A major issue of CiM is the area and power consumption of analog-to-digital converters (ADCs). This work proposes a low-bit A/D conversion system to improve area/energy efficiency. However, the application-level accuracy is degraded due to quantization error and the limited range of low-bit ADC. To determine the optimal ADC range systematically while maintaining application-level accuracy, Layer Input/Output (I/O) Range Training (LIORAT) is proposed. LIORAT simultaneously trains the weights of a NN and the I/O range of each NN layer. Additionally, a digital ReRAM look-up table (LUT) is placed just after the ADC in the proposed A/D conversion system. Digital ReRAM LUT is used for non-MAC operations in the NN, such as batch normalization (BN). The values of the LUT are uniquely determined by the BN parameters and I/O ranges obtained by LIORAT. The application-level accuracy degradation caused by the ADC non-linearity and ReRAM weight errors can be compensated only by retraining BN parameters. Hence, the accuracy is recovered by updating digital L UT with the retrained BN parameters. Weight error compensation by updating digital L UT requires lower write accuracy compared to analog weight rewriting. ResNet-32 trained with the proposed LIORAT achieves 87.1 % inference accuracy on the CIFAR-10 dataset with only 10% LUT area overhead, 4-bit weights, 2-bit DAC, and 4-bit ADC. By updating the L UT, the magnitude of the tolerable error is more than doubled compared to the case without compensation.