A-SARSA: A Predictive Container Auto-Scaling Algorithm Based on Reinforcement Learning

Due to the lightweight and flexible characteristics, containers have gradually been used for the application deployment and the basic unit for resource allocation in a cloud platform recently. Reinforcement learning (RL), as a classic algorithm, is widely used in virtual machine scheduling scenarios due to its advantages of adaptability and robustness. However, most RL methods have problems in container scheduling, such as untimely scheduling, lack of accuracy in decision-making and poor dynamics that will lead to a higher SLA violation rate. In order to solve the above problems, a predictive RL algorithm A-SARSA is proposed, which combines the ARIMA model and the neural network model. This algorithm not only ensures the predictability and accuracy of the scaling strategy, but also enables the scaling decisions to adapt to the changing workloads. Through a large number of experiments, the timeliness and effectiveness of the A-SARSA algorithm for container scheduling are verified, which can reduce the SLA violation rate dramatically while keeping the resource utilization rate at a good level.