An Algorithm Controlling Response Time of Multi-Service Application Based On Bayesian Optimization

In the research field of cloud auto-scaling, controlling application-level response time through resource provision to meet the predefined threshold is vital. But there still exists many challenges in the multi-service application scenario. This paper first points out that, in a multi-service application, increasing or decreasing the instance number of one service may affect other services' response time. This phenomena brings more complexity in designing an auto-scaling strategy as the application-level response time is constructed and affected by the time of every service in it. To hide those affections among internal services, the change in application-level response time under various services' instance number configurations is modelled as a black-box optimization problem. Then, an improved Bayesian Optimization approach is proposed to resolve it. As the improved part, a heuristic method will dynamically and gradually expand the search space according to each service's response time. So a group of services can be scaled simultaneously and more than one instance can be added into a service at one scaling action. The experiments results conducted on the popular kubernetes platform indicate that our method is faster than other traditional ones in controlling the application-level response time down to the predefined threshold value.