Luthier: A Dynamic Binary Instrumentation Framework Targeting AMD GPUs

Dynamic Binary instrumentation (DBI) is a widely used technique for collecting detailed, fine-grained information from program execution without requiring recompilation or access to the program's source code. DBI provides several benefits over static instrumentation, including full code discovery and the ability to selectively toggle profiling during runtime. Luthier is an open-source DBI framework targeting AMD GPUs, designed to integrate and run seamlessly on the ROCm software stack. During runtime, Luthier allows inspection of loaded GPU code objects and carries out instrumentation by either manually modifying instructions or inserting calls to special device functions (i.e., 'hooks'') at user-specified locations in the program. Luthier hooks allow inspection and modification of the device visible state, and can communicate with the host via host-accessible device memory buffers. Luthier also supports switching between instrumented and un-instrumented versions of a kernel. In this paper, we describe some of the key design challenges we encountered when developing this open-source DBI framework. We then showcase Luthier's user-facing APIs and internal components, providing example usecases implemented using our framework. While Luthier incurs a 50X runtime overhead (on average) when running an instrumented application, this overhead is 10 times lower as compared to the state-of-the-art GPU-based DBI framework, when running equivalent tools on the same workload written in CUDA.