COCOSSim: A Cycle-Accurate Simulator for Heterogeneous Systolic Array Architectures

Performance modeling is an essential tool for enabling cost-effective and efficient exploration of architectural and microarchitectural design decisions for hardware development. However, existing simulators face limitations in architectural flexibility, memory hierarchy modeling, and support for modern accelerators that cater to the computational demands of contemporary machine learning models, where non-linear operations like softmax and other vectorized activations have become increasingly important. To address these gaps, we present COCOSSim, a cycleaccurate performance simulator designed for evaluating architectural and microarchitectural modifications in heterogeneous systolic array-based accelerators. COCOSSim supports a wide range of neural network inference workloads, integrating systolic arrays with vector units for non-linear operations and DRAMSim3 for memory modeling. By offering a PyTorch frontend for seamless model integration, flexibility in scheduling strategies, and the ability to make architectural and microarchitectural modifications, COCOSSim enables detailed performance analysis and design exploration. When validated against the Google TPU v3, COCOSSim achieves an average error rate of 13 %. Additionally, COCOSSim is highly scalable and offers simulation speeds an order of magnitude faster than prior work while providing significantly greater flexibility and enhanced modeling capabilities. We present two case studies demonstrating how COCOSSim can be used to evaluate architectural modifications to modern heterogeneous accelerators: model parallelism and operation fusion.