Developing an Open-Source Software Stack for On-Premise GPU Resource Management for Teaching in Artificial Intelligence and Deep Neural Networks

In the realms of computing and education, numerous tasks have transitioned to cloud-based solutions. Often, this shift represents the most convenient and cost-effective approach. Many cloud providers extend complimentary services to students and educational institutions, fostering learning and experimentation. However, a significant limitation emerges in scenarios requiring compute-intensive calculations, especially those dependent on graphical processing units (GPUs). Although cloud-based services offering higher compute power exist, their cost can be prohibitively high. In numerous instances, an on-site server emerges as a more economical alternative. However, such setups necessitate a software stack capable of managing resource reservations and allocations. Our survey of existing solutions revealed a common drawback: the necessity for multiple licenses, which undermines cost efficiency. To address this challenge, we propose an innovative solution: an open-source software stack designed to efficiently manage these needs without requiring any licensing fees to external companies or vendors. This paper details the architecture and design decisions underpinning this novel system.