AI Accelerator Embedded Computational Storage for Large-Scale DNN Models

As the model size of Deep Neural Networks (DNN) is extremely expanding, especially in Natural Language Processing (NLP) field, it is not suitable for performing AI application on conventional DRAM-based systems that lack capacity. For such a reason, we propose a new computational storage by integrating a Neural Processing Unit (NPU) that could perform the DNN inference to the NAND Flash Memory Controller (FMC). Our NPU is able to use weight scheduling to efficiently load data from multiple NAND channels, and the optimal number of MAC units in NPU Core module is derived. We evaluate our proposed NPU architecture when using a very large size of DNN model. In contrast to DRAM-based memory system, ours show an equal inference performance, and total memory access power and latency are reduced by 22.6% and 36.3%, respectively. Also, by using the data buffer of NAND flash memory as shared SRAM, the area overhead could be reduced by 82% compared to when NPU using dedicated SRAM.