Connectionless port scan detection on the backbone

Considerable research has been done on detecting and blocking portscan activities that are typically conducted by infected hosts to discover other vulnerable hosts. However, the focus has been on enterprise gateway-level intrusion detection systems where the traffic volume is low and network configuration information is readily available. This paper investigates the effectiveness of existing portscan detection algorithms in the context of a large transit backbone network and proposes a new algorithm that meets the demands of aggregated high speed backbone traffic. Specifically, we evaluate two existing approaches - the portscan detection algorithm in SNORT, and a modified version of the TRW algorithm that is a part of the intrusion detection tool BRO. We then propose a new approach, TAPS, which uses sequential hypothesis testing to detect hosts that exhibit abnormal access patterns in terms of destination hosts and destination ports. We perform a comparative analysis of these three approaches using real backbone packet traces, and find that TAPS exhibits the best performance in terms of catching the maximum number of true scanners and yielding the least number of false positives. We have a working implementation of TAPS on our monitoring platform. Further implementation optimizations using bloom filters are identified and discussed.