Labeling Cloud Metrics Data for Fault Detection in Cloud Using Active Learning With Test Suite

Ensuring the quality of service of applications deployed in inherently complex and fault-prone cloud environments is of utmost concern. While machine learning based fault management solutions help attain the desired reliability, they require labeled cloud metrics data for training and evaluation. Furthermore, high dynamicity of cloud environments brings forth emerging data distributions, which necessitate frequent labeling of data for model adaptation. We propose a test suite-based active learning framework for automated labeling of cloud metrics data with the corresponding cloud system state while accounting for emerging fault patterns and data or concept drifts. We have implemented our solution on a cloud testbed and introduced various emerging data distribution scenarios to evaluate the proposed framework’s labeling efficacy over known and emerging data distributions. According to our results, the proposed framework achieves about 41% higher weighted F1-score and 34% higher average Area Under the One-vs-Rest Receiver Operating Characteristic Curve (AUC) score than a system without any adaptation for emerging data distributions.