Scalable Multi-Model Orchestration in AI Microservices with Kubernetes and Serverless for Event-Driven MLOps Pipelines

The ability to efficiently orchestrate multiple machine learning models will be crucial in this ever-changing landscape of AI for deploying scalable and resilient applications. This paper presents a holistic framework for scalable multi-model orchestration within the architecture of AI microservices-a Kubernetes and serverless-driven, event-driven pipeline in MLOps. The proposed framework integrates Kubernetes, which manages container orchestration with robust scalability, automated deployment, and seamless management of microservices. For this, serverless computing supports dealing with on-demand resource allocation, less operational overhead, and cost-effectiveness. This pipeline is further optimized by an event-driven paradigm, where events generated from real-time data streams trigger the model workflows to assure responses in time and relevance. We will further demonstrate the applicability of the proposed framework on a publicly available dataset, CIFAR-10, and show how diverse models for image classification can be orchestrated on one platform to performant and reliable results in real-world scenarios. These experiments yield a reasonable improvement in the speed, scalability, and operational efficiency at deployment, proving the potential of combining Kubernetes and serverless for advanced MLOps implementations.