Online feature selected semi-supervised decision trees for network intrusion detection

Network intrusion detection systems need to detect abnormal behaviour in network data as soon as possible and with as little user intervention as possible. In this paper, we describe a semi-supervised network anomaly detection system. Our system uses online clustering to summarize the available network data. Clusters are represented using extended cluster features that comprise of not only features related to the original features, but also features that describe the relationships between clusters. Each cluster is labeled by the user as anomaly or normal and then a decision tree is trained based on this information. The incoming new data is labeled according to the output of the decision tree. We show that this system achieves much better performance than an unsupervised anomaly detection system. We also show that using online feature selection on the cluster features reduces the decision tree complexity without hindering the accuracy.