Using Partial Correlation for Effective Metric Selection and Anomaly Detection in Multi-tier System

Large-scale data centers allow organizations to gain access to computer resources without incurring high costs in purchasing and maintaining IT infrastructure. In these environments, due to the large number of hardware and software involved, anomaly detection is difficult but essential for service provisioning. Computer systems hosted in data centers usually involve multiple layers and provide a large set of metrics for tracking their operation. The analysis of all available metrics generates drawbacks associated with communication, storage and processing. A more efficient way to support anomaly detection and minimize the cost of monitoring is to use stable statistical correlations among metrics that reflect the system state. We present the strategies PCTN, MST-PCTN and PCTN-MST, based on partial correlation, for selecting metrics to support anomaly detection in multi-tier systems. We evaluate the proposed strategies using an e-commerce, Web transaction benchmark. Results show that the PCTN-MST strategy allowed the construction of a monitoring network with 8% less metrics than that obtained with MST and achieved a fault coverage up to 10% larger.