Intruder detection based on graph structured hypothesis testing

Anomaly based network intruder detection is considered. In particular, we view anomaly detection as a statistical hypothesis testing problem. The null hypothesis associated with each host is that it is acting normally, while the alternative is that the host is acting abnormally. When considered in relation to the network traffic, these host-level hypotheses form a graphically structured hypothesis testing problem. Some network intrusions will form linked regions in this graph where the null hypotheses are false. This will be the case when an intruder traverses the network, or when a coordinated attack is performed targeting the same set of machines. Other network intrusions can lead to multiple unrelated hosts acting abnormally, such as when multiple attackers are acting more or less independently. We consider model based approaches for detecting these different types of disruptions to the network activity. For instance, network traversal is modeled as a random walk through the network stringing together multiple abnormally acting machines. A coordinated attack targeting a single machine is modeled as multiple anomalous hosts connecting to a randomly selected target. The advantage of modeling the attacker patterns is that, under ideal conditions, this defines an optimal detector of the intruders. This optimal detector depends on unknown parameters, and is therefore less attractive for practical use. We describe pragmatic approaches that, in simulations, achieve close to optimal detection rates. The methodology is applied to a real-world network intrusion, clearly identifying the attack.