A Workload Characterization of the SPEC CPU2017 Benchmark Suite

The Standard Performance Evaluation Corporation (SPEC) CPU benchmark suite is commonly used in computer architecture research and has evolved to keep up with system microarchitecture and compiler changes. The SPEC CPU2006 suite, which remained the state-of-the-art for 11 years was retired in 2017, and is being replaced with the new SPEC CPU2017 suite. The new suite is expected to become mainstream for simulation-based design and optimization research for next-generation processors, memory subsystems, and compilers. In this paper, we extensively characterize the SPEC CPU2017 applications with respect to several metrics, such as instruction mix, execution performance, branch and cache behaviors. We compare the CPU2017 and the CPU2006 suites to explore the workload similarities and differences. We also present detailed analysis to enable researchers to intelligently choose a diverse subset of the CPU2017 suite that accurately represents the whole suite, in order to reduce simulation time.

I. Introduction

Benchmarking is one of the critical stages of the continuous development cycle of compilers and system microarchitectures. A set of representative applications (collectively referred to as ‘benchmark suite’) is executed on the computer system to either identify potential optimization opportunities in the system software or hardware, or to qualitatively determine the benefits of the optimizations performed. There are several benchmark suites available that focus on different application domains. For example, the EEMBC [1] and MiBench [2] benchmark suites focus on embedded system applications in different domains like automation, communication, and office software. The SPLASH [3] and PARSEC [4] benchmark suites, comprise of multi-threaded workloads and evaluate a processor's multi-threading capabilities.