Anomalies in network traffic

We report the results of a search for anomalies in network traffic. Our data set consisted of two billion packets collected over four days at the gateways of our large corporate network. Analysis of the distributions of the packet metadata fields (IP addresses, ports, time-to-live and packet length) revealed anomalous activity including IP scans, port scans and hybrid scans as well as coordinated and synchronous activity. Analysis of such large amounts of data can be onerous; the use of Apache Hadoop to implement reliable, scalable, distributed computing enabled us to perform our computations rapidly on a small cluster of servers.