A Quantitative Approach for Estimating the Scaling Thresholds and Step Policies in a Distributed Microservice Architecture

Microservice architecture (MSA) has become a de facto standard for developing complex web applications lately. Horizontal scalability, domain isolation, agility and the provision to use heterogenous technologies are some of the key factors for the growing popularity of this architecture. To automatically cater to varying load patterns, quite a lot of advancements have been made in the field of cloud computing, containerization and orchestrating mechanisms which aid to perform the auto scaling of the microservices. However, setting up the scaling policies, optimal upper and lower thresholds is a daunting task for large applications. It generally involves some initial guess work followed by multiple rounds of tuning based on the real time load variations. This process causes situations where either the service becomes unavailable to the load when the thresholds are on the lower side, (or) underutilization of the compute resources when they are on higher side. This paper aims to find a quantitative way of determining the thresholds and step-up policies by deducing the mathematical formulas. To solve this formidable problem, we propose a model in which the total resource consumption of a container running in the peak load scenario can be calculated by - (1) first identifying the critical transactions and their maximum concurrency rates,(2) then calculating the resource consumption of such transactions in a controlled environment and (3) finally applying those values to the mathematical formulas based on Gaussian functions to calculate the total resource consumption for the peak load scenario. Using the total resource consumption value and considering the network and startup latencies, an optimal upper threshold value for step-up functions can be calculated. In this paper, we calculated the upper threshold values using the above-mentioned approach and verified using a research project that the calculated value is indeed the minimum number of containers to handle load.