AdaptPerf: On Adaptive and Scalable Computing Power Measurement for Heterogeneous Devices

Computing power measurement faces the challenges of heterogeneity of devices, poor scalability with workloads, and discrepancies between measurement methods and models. In this article, we propose a computing power measurement framework called AdaptPerf to address these issues. AdaptPerf uses the technique of neural architecture search to build a hypernetwork of benchmark models for heterogeneous devices and adaptively selects the ones suitable for them. It achieves equivalency in computing power measurement through kernel-level latency analysis. We further prove that the adaptive benchmark measurement model selection problem (ABMSP) is an NP-hard problem and introduced a deep reinforcement learning approach to solve it. The theoretical analyzes show that the proposed method can guarantee that the obtained solution approximates the Pareto frontier. We implement AdaptPerf in a real-world environment, constructing a hypernetwork of over $4.2 \times 10^{20}$ models for heterogeneous devices to autonomously select the benchmark model adapted to the devices. We use different tools, such as Nsight Compute, Nsight System, to analyze and evaluate the feasibility and performance of AdaptPerf. It is found that the models generated by AdaptPerf increase computational throughput by 46.63% and improve data reuse rate by 93.7%, compared with more than 60 traditional models. AdaptPerf also exhibits superior load adaptability and model complexity compared with existing methods, such as MLPerf.