PC2A: Predicting Collective Contextual Anomalies via LSTM With Deep Generative Model

Proactive anomaly detection and diagnosis play an essential role in ensuring the security and stability of a large-scale information technology (IT) system with thousands or even millions of components that are interacting with each other. Collective contextual anomalies (CCAs) carry the characteristics of both collective and contextual anomalies. This type of anomalies is common in IT system monitoring, often manifested as security risks to be ameliorated, service outages to be eliminated, or stragglers to be mitigated. However, most existing studies emphasize primarily on the detection of point anomalies while the prediction or early detection of CCA has been an underexplored topic. In this paper, we propose a framework for discovering and studying CCAs in multiple time series based on a combination of semi-supervised deep learning, time series modeling, and graph analysis. A primary advantage of the proposed framework is that it can effectively predict CCAs with no human intervention. In addition, the performance of the proposed method can be further enhanced via learning from a small amount of anomalous training data, if it is available. Finally, the proposed framework is of low computational complexity and is thus particularly suitable for large-scale data streams. Simulation studies are carried out to reveal the superior performance of the proposed method and underscore the significant benefits of combining deep neural networks with time series analysis and graph models for the prediction and analysis of CCAs.