Autoscaling Based on Response Time Prediction for Microservice Application in Kubernetes

Containerized application are evolving along with the microservice architectures in distributed application development. This trend shows the importance of managing and orchestrating containerized applications thus applications can operate properly. One of the aspects of container orchestration is scaling or increasing the application’s ability to handle more requests. In this study, an autoscaler based on response time prediction is developed for microservice applications in Kubernetes environment. The prediction function is developed using a machine learning model that features performance metrics at the microservice and node levels. The response time prediction is then used to calculate the number of pods required by the application to meet the target response time. Our experiment shows that the proposed autoscaler can serve more requests that match the target response time compare with the Kubernetes Horizontal Pod Autoscaler (HPA) that are using CPU usage as the target. However, as the consequence, the proposed autoscaler consumes more resources than the Kubernetes HPA.