A Profiling-Based Benchmark Suite for Warehouse-Scale Computers

Benchmarking warehouse-scale compute (WSC) systems poses unique challenges due to their scale, workload diversity, and focus on data-intensive computation. Traditional benchmarks often fall short in capturing the nuances of WSC behavior. Creating a public benchmark suite that is representative of the workloads used by actual warehouse-scale computers is challenging, as they typically run proprietary, non-public software that operates on confidential data. In this paper, we present Fleetbench, a benchmark suite that is representative of the workloads used at Google. Fleetbench does not benchmark entire applications; rather, it focuses on common building blocks, known as the “datacenter tax”, that are at the core of a wide range of different data center applications, and many of which are open source. The relevant building blocks are selected based on fleet-wide profiling data. Representative input data is collected using a fleet-wide value profiler, ensuring that no confidential information is revealed. Fleetbench is available as open source on Github²¹https://github.com/google/fieetbench.