An Enhanced Data Cache with In-Cache Processing Units for Convolutional Neural Network Accelerators

Convolutional neural network algorithms rely on large quantities of data to extract hidden information. In order to achieve good performance, a CNN accelerator should be able to easily access a wide range of memory space. However, on-chip memory has limited capacity, and needs to be manually managed by the accelerator controller. Besides, accessing off-chip memory like a DRAM array needs a physical address, which is not calculated until a memory access request arrives at a memory access control unit. Managing those issues manually complicates CNN accelerator design. Therefore, we propose a data cache enhanced with in-cache processing units, which processes segment address in parallel with data, and exploits data locality in an automatic manner. Overheads such as physical address translation, quantization, and ReLU, are offloaded from the accelerator controller.