Predictive and Prescriptive Analytics for Performance Optimization: Framework and a Case Study on a Large-Scale Enterprise System

In any industrial or software system, predicting future values of measurable parameters well in advance is of utmost importance for avoiding disruptions. The historical data on system parameters measured at regular time intervals can be leveraged to address this long horizon prediction problem. However, complex interdependencies between the parameters and the need for avoiding false recommendations pose challenges in this prediction task. An equally challenging and useful exercise is to identify the 'important' parameters and optimize them in order to attain good system performance. This paper describes a generic framework, along with specific methods, for this data analytics problem and presents a case study on a large-scale enterprise system. The proposed method combines techniques from machine learning, causal analysis, time-series analysis and stochastic optimization to achieve accurate prediction (estimating future values of parameters) and reliable prescription (controlling independent parameters to optimize system performance). The approach is validated with data from a large-scale enterprise service bus consisting of about 30 parameters measured at 5 minute intervals over a period of 6 months.

I. Introduction

Many industrial processes as well as software applications have several measurable parameters associated with them. Recent technological advancements have not only enabled efficiently recording and monitoring the values of these parameters, but also predicting their future values based on historical data. This can be useful in avoiding system failures and outages that lead to financial loss and/or customer dissatisfaction.