Evaluation of Anomaly Detection Algorithms Made Easy with RELOAD

Anomaly detection aims at identifying patterns in data that do not conform to the expected behavior. Despite anomaly detection has been arising as one of the most powerful techniques to suspect attacks or failures, dedicated support for the experimental evaluation is actually scarce. In fact, existing frameworks are mostly intended for the broad purposes of data mining and machine learning. Intuitive tools tailored for evaluating anomaly detection algorithms for failure and attack detection with an intuitive support to sliding windows are currently missing. This paper presents RELOAD, a flexible and intuitive tool for the Rapid EvaLuation Of Anomaly Detection algorithms. RELOAD is able to automatically i) fetch data from an existing data set, ii) identify the most informative features of the data set, iii) run anomaly detection algorithms, including those based on sliding windows, iv) apply multiple strategies to features and decide on anomalies, and v) provide conclusive results following an extensive set of metrics, along with plots of algorithms scores. Finally, RELOAD includes a simple GUI to set up the experiments and examine results. After describing the structure of the tool and detailing inputs and outputs of RELOAD, we exercise RELOAD to analyze an intrusion detection dataset available on a public platform, showing its setup, metric scores and plots.

I. Anomaly Detectors

Cyber-physical infrastructures or Systems of Systems are composed of many different software layers and a multitude of services. Due to the complexity, dynamicity and governance of these systems, instrumenting each individual service for monitoring purposes and characterizing all the possible errors or attack that may manifest is often not feasible [23], despite being widely acknowledged as crucial. To such extent, anomaly detection [1] was proposed; it deals with the problem of finding patterns in data that do not conform to the expected behavior.