Designing an Improved ML Task Scheduling Mechanism on Kubernetes

As the fields related to machine learning (ML)/deep learning (DL) continue to mature, the MLOps machine learning automation process is also gradually emerging. Then, many open-source MLOps frameworks based on Kubernetes have begun to be proposed. Currently, most Kubernetes-based MLOps frameworks aim to establish a common and easy-to-use ML pipeline environment for users to use based on ML containerized tasks. However, Kubernetes’ default container scheduler only considers the resource conditions of individual containerized tasks, rather than considering the scheduling of the entire containerized ML task composition. Such a situation may lead to the system resources not being utilized properly. Therefore, this study designs an improved ML task mechanism based on the Kubernetes-based platform to replace the Kubernetes default scheduler. The scheduling strategy in Kubernetes can be modified to better suit the needs of the machine learning development environment.