Intelligent Autoscaling of Microservices in the Cloud for Real-Time Applications

Cloud applications are becoming more containerized in nature. Developing a cloud application based on a microservice architecture imposes different challenges including scalability at the container level. What adds to the challenge is that cloud applications impose quality of service (QoS) requirements and have various resource demands requiring a customized scaling approach. For example, real-time applications require near real time response time as a QoS. Existing autoscaling technologies such as Kubernetes offer some customization to a set of threshold values for autoscaling. The challenge is identifying the right values for the different autoscaling parameters that will guarantee QoS in a changing dynamic environment. Advancements in machine learning and reinforcement learning (RL) provides a means for autoscaling in cloud applications with no domain knowledge. In this article, we introduce an intelligent autonomous autoscaling system for microservices autoscaling in the cloud with QoS constraints. The system consists of two modules. The first module identifies the microservice resource demand via a generic autoscaling algorithm deployed on the Google Kubernetes Engine (GKE). Our algorithm adapts the Kubernetes autoscaling paradigm based on the application resource requirements. The second module uses reinforcement learning agents to learn and identify the autoscaling threshold values based on the resource demand and QoS. Experimental results show an enhancement in the microservice response time up to 20% compared to the default autoscaling paradigm. In addition, the RL agents can identify the autoscaling threshold values while maintaining a response time below the QoS constraint. Our proposed work provides a customized autoscaling solution for microservices in cloud applications while adhering to QoS constraints with minimum user interaction.