3-D Stacking for AI Systems: Emulating Heterogeneous 3-D Architectures for AI

Three-dimensional integration provides substantial benefits in boosting system performance and efficiency, especially in the end-of-scaling era. The increasing demand for greater computational power, larger GPU/cache sizes, energy efficiency, and low-power custom artificial intelligence (AI) hardware further drives the need for 3-D integration. Despite the demonstrated advantages of 3-D technology, such as enhanced interconnectivity and improved performance across various applications, the design of heterogeneous 3-D systems experiences challenges. Among these, the issues related to temperature management and long-term reliability arise from the complex interactions among system components, which are difficult for current modeling tools to accurately capture, necessitating detailed hardware characterization. This study explores the recent drivers of 3-D integration and highlights the growing need for hardware emulation frameworks. It presents an emulator design focused on profiling key metrics such as power, temperature, noise, interlayer bandwidth, and lifetime reliability, enabling the emulation of different stacking configurations. This framework not only allows for macro-level control of activity levels but also incorporates a specialized sensor infrastructure to evaluate heat distribution, interlayer noise, power delivery, reliability, interconnectivity, and the interactions among crucial design objectives.