Modeling Chiplet-to-Chiplet (C2C) Communication for Chiplet-based Co-Design

Chiplet-based processor design, which combines small dies called chiplets to form a larger chip, enables scalable designs at economical costs. This trend has received high attention such that standards for chiplet design have rapidly established, including packaging, protocols, and Chiplet-to-Chiplet (C2C) interfaces. With numerous well-defined chiplet options available, hardware architects would leverage on the co-design process to make optimal decisions on design parameters. An important performance limitation in multi-chiplet designs come from the protocol translation in the C2C communication, needed to maintain cache coherency and avoid risk of deadlocks. When integrating multiple chiplets, deadlocks can happen from both protocol and routing, making deadlock-free designs important. This paper tackles these challenges by introducing a Chiplet-to-Chiplet Gateway (C2CG) architecture, a C2C interface that bridges two chiplet protocols and ensures deadlock-free C2C communication. We also extend the Coherent Hub Interface (CHI) protocol to support cache coherent data sharing among cores across chiplets. The complete design is implemented in the gem5 simulator, constructing a modeling tool for chiplet-based co-design targeting next-generation High-performance Computing (HPC) processors. We demonstrate the benefit of the model through a design space exploration of three 64-core Armv8 HPC processor configurations: monolithic, two- and four-chiplet. The exploration, using representative HPC benchmarks, provides insights into C2C parameters and studies the impact of Non-Uniform Memory Access (NUMA) configuration, giving valuable co-design feedback for hardware architects.