Simulator calibration for accelerator-rich architecture studies

Cycle-accurate simulators are widely used in architecture research. A certain degree of performance inaccuracy is expected in such simulators. Such mismatches are tolerable in micro-architectural studies that emphasize relative performance. However, when modeling accelerator-rich heterogeneous systems, an inaccurate baseline CPU model can lead to severe over/underestimation of the speedup and/or energy savings. In this paper, we present a systematic approach and methodology for calibration of cycle-accurate simulators targeting such studies. We further compare the potential impact that an unrepresentative baseline CPU model can have on heterogeneous system design and describe how it could cause to misleading design evaluations. We demonstrate our approach on calibration of MARSSx86, a widely-used, cycle-accurate x86 system simulator. Using our methodology, we calibrate MARSSx86 to closely match the performance of state-of-the-art Intel machines targeting high-performance computing (HPC) benchmarks. Our calibrated MARSSx86 shows on average less than 10% error across a wide range of HPC and general-purpose benchmarks. Using this calibrated baseline simulator, we further quantify the impact of such calibration on an accelerator-rich architecture case study. Our results show that an accelerator estimated to obtain a speedup of 10.3x using unrefined simulators can in reality only achieve a 5.3x speedup when an accurate CPU model is employed.