Efficient Task Scheduling and Allocation of GPU Resources in Clouds

The growing demand for computational power in clouds is leading to the increasing adoption of Graphics Processing Units (GPUs). However, due to the high cost of GPUs, it is necessary to allocate these GPU resources efficiently. Cloud constraints such as isolation and multiple tenancy need also to be taken into account. An example of a challenge is how to avoid under-utilization when allocating resources under the constraint of isolation. Yet another example is how to schedule tasks and allocate GPU resources to them while respecting task requirements (e.g. completion deadlines) and fairness between the tenants. Unfortunately, none of the existing work satisfactorily addresses these challenges to the best of our knowledge. This paper proposes an algorithm for efficient GPU resource allocation in clouds. The algorithm relies on GPU multitasking methods supported by both hardware and software. It schedules tasks and allocates resources to the tasks efficiently while respecting isolation, fairness and task requirements. According to experimental results, our proposed algorithms outperform state-of-the-art solutions by up to 24.25% for the maximum normalized task completion time while significantly reducing GPU resource usage.