schedGPU: Fine-grain dynamic and adaptative scheduling for GPUs

In order to obtain maximum throughput (i.e. jobs/second) and maximum hardware utilization in a highperformance computing (HPC) scenario, it is necessary to use proper scheduling of the different jobs among the available resources. In the case of heterogeneous platforms combining both CPUs and GPUs, an entire job could be delayed because of the lack of GPU resources (e.g. GPU memory), while CPU resources may remain idle. For certain kinds of applications, a fine-grain scheduling, dynamically blocking and releasing resources, and capable of adapting quickly to the changing requirements, could be the perfect solution. In this paper we present an implementation for this solution in the form of a middleware called schedGPU. The performance evaluation, using a financial application for options pricing, clearly shows its benefits: the throughput is doubled, while the hardware utilization is significantly increased.