mTADS: Multivariate Time Series Anomaly Detection Benchmark Suites

Detecting anomalous events in time series data, ranging from manufacturing processes to health care monitoring, is important. The problem of uncertainty in real-world datasets and its anomalies makes it challenging to validate and compare results between algorithms. With this in mind, we present two benchmark suites to fill gaps in today’s landscape of datasets for anomaly detection in multivariate time series data. Here, one suite focuses only on fully synthetic sequences to provide a playground for testing algorithms with complete knowledge of the sequences. The second suite bridges between fully synthetic and real-world sequences. It provides a few extensive sequences with close-to-reality complexity but synthetic injected anomalies. The paper provides a detailed overview of the suites content and complexities. It further includes a concise overview and showcases the strengths and weaknesses of 34 algorithms in the evaluation. The benchmark suites highlight issues regarding algorithms and metrics and should support new research directions.