Virtualized GPUs in High Performance Datacenters

Virtualization has become the foundation of high performance datacenters. The scale of high performance datacenters is similar to the scale of supercomputers used in High Performance Computing (HPC). Both involve millions of CPU cores, tens of thousands of accelerators like GPUs, FPGAs connected by high speed interconnects and different connection topologies. GPUs are essential in high performance computing because of their high throughput and energy efficiency. Many high performance computing workloads have avoided virtualization because of performance concerns. As the scale of high performance datacenter increases, the need for ease of management becomes critical. The virtualized GPUs offer new opportunities in efficient resource allocation in managing datacenters. In this paper, we present different GPU virtualization solutions and discuss its related aspects that helps GPU based applications running efficiently at scale. We also present a comprehensive performance study of virtualized GPU that addresses main considerations of running high performance applications in virtualized datacenters. We evaluate the performance of virtual GPUs from many different angles: the overhead of virtual GPUs compared to physical GPU, the impact of using container, the use of virtualized GPUs for many different types of HPC applications, the solution of mixing workloads to increase resource utilization and system consolidation, the scalability, as well as how to make the right choice of vGPU profile.