Real Time Failure Prediction of Load Balancers and Firewalls

Load balancers and firewalls are entry points to many key websites in any network infrastructure. Any downtime due to these devices, even for a few minutes, may result in major business impact. There are many proactive event management systems/products which can monitor their health and alert on device failure. However, predicting failure with sufficient lead time still remains a challenging problem. Event management systems take system log messages and simple network management protocol (SNMP) traps as inputs and typically use rule based framework to generate network events. In this paper we discuss how we can use event data alone to build predictive model for device failures. Here all event data is generated by an event management tool. Event data volume is substantially less compared to full system log data. Also, these network devices fail rarely. Lower volume of event data enables us collect historical data for longer durations and thus collect decent number of failure samples over time which we can use for training and validation. We present a prediction model for failure events based on event sequence data. We have introduced new stratified sampling techniques along with a new feature engineering method using sliding time windows on event data. Experiments show that for rare device failure events like the load balancers it suffices to use event data to model device failures instead of using raw system log data. We have evaluated binary classification algorithms like support vector machines (SVM) and logistic regression (LR). Experimental results show that with the proper noise cleanup technique and model tuning we can achieve precision of 77% and recall of 67% for the failure predictions of network devices only from event data.