Filer Response Time Prediction Using Adaptively-Learned Forecasting Models Based on Counter Time Series Data

Ability to predict the future performance of a storage system is critical for its efficient management. A modern storage system is a very complex combination of hardware and software elements and inferring its state from those of its individual components is practically impossible. Moreover, the state of the system undergoes continuous changes due to diverse and evolving workloads, frequent configuration changes and natural degradation in components. This paper presents an approach to predict the future response time of a storage system by applying a combination of machine learning techniques on the data of its internal parameters. To the best of our knowledge, this is the first attempt to quantify the response time of a storage system directly from the time-series of its internal parameters. We present a systematic experimental setup for collecting the system data and for measuring its response time while loading the system with different standard workloads. Another novel contribution of the paper is a combination of time-series forecasting model followed by a regression model to predict the response time repeatedly in real-time. The said model is shown to be able to repeatedly predict fairly accurate 15-minutes-ahead response time values using the system data of the past 20-minutes' window. It is shown that the validation calculations typically complete within ~30 seconds on an average using a computer of moderate configurations making the presented model applicable for practically reasonable prediction horizons. The design of the model is such that it can be adaptively tuned online at regular intervals as more system data is observed.