Battle of Compilers: An Experimental Evaluation Using SPEC CPU2017

In order to meet growing application demands, modern processors are constantly evolving. Today they integrate multiple processor cores, an on-chip interconnect, large shared caches, specialized hardware accelerators, and memory controllers on a single die. Compilers play a key role in bridging the gap between abstract high-level source code used by software developers and the advanced hardware structures. This paper reports the results of a measurement-based study that evaluates three most prevalent compilers used in industry and academia. We compare the effectiveness of the Intel Parallel Studio XE-19 (IPS), the LLVM Compiler Infrastructure project, and the GNU Compiler Collection using the SPEC CPU2017 benchmark suite that is widely used to evaluate performance of modern computer systems. We quantitively evaluate the compilers with respect to metrics such as benchmark build times, executable code sizes, and execution times. The benchmarks are compiled using comparable optimization levels and they are run on an Intel 8th generation Core i7-8700K. The results show that LLVM creates the smallest executables, GNU has the lowest build times, and IPS has the best performance.