Anomaly Detection in Distributed Systems via Variational Autoencoders

Distributed systems have been widely used in the information technology industry. However, with the increasing scale and complexity of distributed systems, the efficiency and accuracy of manual anomaly detection in system logs have decreased. Therefore, there is a great demand for a highly accurate and efficient automatic anomaly detection method based on system log analysis to ensure the reliability and the stability of large-scale distributed systems. In this paper, we propose VeLog, an automatic anomaly detection method based on variational autoencoders (VAEs). In the offline training phase, VeLog learns the patterns of normal log sequences and then generates normal intervals. In the online detection phase, VeLog detects an anomaly by automatically evaluating whether the distance between the input vector and its estimated vector matches these normal intervals. We evaluate VeLog on log datasets collected from representative distributed systems. The experimental results demonstrate that VeLog can detect anomalies with high accuracy and good efficiency.