Application-aware Resource Sharing using Software and Hardware Partitioning on Modern GPUs

Graphic Processing Units (GPUs) are known for the large computing capabilities they offer users compared to traditional CPUs. However, the issue of resource under-utilization is becoming more apparent as more and more applications are unable to saturate modern GPUs which have even higher processing capabilities. While concurrency mechanisms like hardware partitioning have resulted in better utilization compared to deployments without sharing, the issue of resource under-utilization still persists even in deployment scenarios where applications are executed on the smallest GPU partitions of modern GPUs. Software partitioning on the other hand, does not guarantee isolation during executions leading to issues of interference and consequently limiting the number of applications which can be run concurrently. Leveraging both software and hardware resource partitioning schemes in an effort to mitigate resource under-utilization issues is yet to be fully explored. In this paper, we evaluate the predictions of a proposed linear regression model relative to actual executions. The results of our experiments show that whilst our approach accurately estimates performance for sharing differently-sized GPU partitions among diverse applications based on each application’s characteristics, it also improves utilization and reduces resource wastage.