QUARK: A Framework for Quantum Computing Application Benchmarking

Quantum computing (QC) is anticipated to provide a speedup over classical approaches for specific problems in optimization, simulation, and machine learning. With the advances in quantum computing toward practical applications, the need to analyze and compare different quantum solutions is increasing. While different low-level benchmarks exist, they often do not provide sufficient insights into real-world application-level performance. We propose an application-centric benchmark method and the QUantum computing Application benchmaRK (QUARK) framework to foster the investigation and creation of application benchmarks for QC. This paper establishes three significant contributions: (1) it makes a case for application-level benchmarks and provides an in-depth "pen and paper" benchmark formulation of two reference problems: robot path and vehicle option optimization from the industrial domain; (2) it proposes the open-source QUARK framework for designing, implementing, executing, and analyzing benchmarks; (3) it provides multiple reference implementations for these two reference problems based on different known, and where needed, extended, classical and quantum algorithmic approaches and analyzes their performance on different types of infrastructures.