Beyond Exascale: A Paradigm shift for AI and HPC

The increasing demand for high-performance computing (HPC) and artificial intelligence (AI) has given rise to a new class of exaflop systems (Fig 1). This has been enabled by the graphics processing unit (GPU) delivering petaflops per socket. Along with the GPU, the continued progress of process technology, advanced 2D and 3D packaging, high bandwidth memory, and network topologies have enabled the current exaflop systems [1]. We introduce a CPU + GPU architecture with a system scale integration framework and key exponential technologies to complement Moore’s law to extend HPC and AI beyond exascale.