Study of container-based JupyterLab and AI Framework on HPC with GPU usage

Container technologies enable portability and reproducibility for applications. A containerized application or service can be quickly prototyped, built, and deployed. Artificial Intelligence (AI) workloads with large datasets are driven by High Performance Computing (HPC) with Graphics Processing Unit (GPU) usage to expedite solving more complex real-world problems. The most widely used container technology is Docker, but due to the root access needed to run Docker containers and related security issues, it cannot be easily adopted in secure HPC systems. Singularity meets the needs of the HPC environment and permits the use of identical Docker containers without alterations to address security issues. This work presents an approach for execution of container-based JupyterLab and AI framework with workload manager using user credentials on HPC with GPU usage. We provide comprehensive performance evaluations of JupyterLab and AI framework containers transformed from Docker to Singularity to solve more complex real-world problems with GPU in an HPC environment in the context of native.