The development of a scheduling system GPUSched for graphics processing units

Single unified programing interfaces and languages like CUDA and OpenCL have been released allowing algorithms across all fields of application to be coded and executed on the graphics processing units. In modern computer systems, the GPU is a processing device accessed through a host and it is not able to run its programs autonomously as a process in the operating system on the advertising host. The host administrates the work on the device including memory transfers, the context switch in the host process to access the device and the launch of kernels (GPU programs). With more and more software featuring GPU support, the access to this shared device has to become more controllable on a system-wide level. Therefore, a better integration of these devices into the operating system including central administration will be necessary. This work takes a first step to create such a central administration for the graphics processing units, by developing a scheduling system called GPUSched. It allows the programmer to formulate different tasks, but then it takes over and manages the execution. The tasks represent GPU code of programs running on a general-purpose operating system. Using a set of microbenchmarks and applications on a GPU system, we show that achieving better utilization of the GPU resources and using these as coprocessors to handle different tasks involve realistic trade-offs.