Evaluating the Observability of Network Security Monitoring Strategies With TOMATO

Monitoring systems for malicious behavior increasingly requires aggregating and analyzing data from various sources, such as network flows, host logs, and end-point monitoring platforms. However, there’s currently a lack of metrics and methodologies to compute the observability and efficiency of a security monitoring strategy. This manuscript introduces TOMATO (Threat Observability & Monitoring Assessment Tool), which is a platform to evaluate the effectiveness of a security monitoring strategy by exploring both the number of known adversarial techniques that can be detected within a network, along with evaluating the number of false-positives produced by the monitoring strategy. The output produces both an observability score and efficiency score of a set of deployed monitoring techniques, which are evaluated based on the data from the environment, and simulated attacks generated from MITRE ATT&CK. The proposed approach is then integrated into an ELK stack and evaluated on real SCADA devices within the WSU Smart City Testbed.