Monolithic 3D Stackable DRAM

Large Language Models (LLMs) serve as critical foundations for a wide range of generative AI applications, which demand substantial memory and storage. The advent of monolithic 3D stackable DRAM, offering cost-effective vertical scaling akin to 3D NAND, presents a promising solution to this challenge. This article conducts a comparative analysis of a variety of 3D DRAM technologies with capacitor or capacitor-less storage mechanisms, focusing on vertical bitline (VBL) and vertical wordline (VWL) configurations. Key performance indicators, including access energy, latency, and storage density, are evaluated at the bank-level considering the 3D array core and peripheral circuits under the array. Moreover, a 3D DRAM-centric LLM accelerator is proposed and benchmarked against the off-the-shelf GPU platforms with high-bandwidth-memory (HBM) with die stacking of 2D DRAM dies. The benchmark results underscore the potential of 3D DRAM technologies in enhancing the efficiency and scalability of LLMs, heralding a new era of large-scale AI model training and inference.