Automated Resource Scaling in Kubeflow through Time Series Forecasting

With the rapid advent of digitization in the last decade, there is evidence of increased web traffic to applications and, more importantly, load on the underlying technology stack supporting large-scale applications. While failure to address the availability can prove to have a long-term impact, this research study aims to further the auto-scaling of cloud workflows and resources, supporting these applications through advanced forecasting techniques, employing machine learning models like ARIMA, SARIMA, Prophet, and LSTMs to estimate traffic to the application accurately. Large-scale operations are mimicked through Kubeflow, a machine learning platform providing a wide range of capabilities from model development to model inferencing to pipelines, and Prometheus, a time-series database, consuming the forecasted metrics to scale resources and workflows.