Split-and-Merge: Detecting Unknown Botnets

On September 20th, 2016, Mirai struck the Internet in a massive surprise attack. This is just but one example of unknown botnets, not detected nor mitigated at the time they happened. Current anomaly detection techniques can only detect them after they have spread. However such attacks are generally preceded by several stages, including infection of hosts or fingerprinting of devices. Being able to capture this activity would allow their early detection. In this paper, we propose a strategy aimed at the early detection of unknown botnets, which acts by (i) splitting and merging distributed monitoring data and related metrics, (ii) monitoring at the port-level, with a simple yet efficient change-detection algorithm based on a modified Z-score measure. The analysis of port usage is first split over different parts of the network, and then merged to retain only similar anomalies. This ensures detection of large-scale attacks and drastically reduces false positives. We validate the approach on the MAWI data set, which provides daily traces of a transpacific backbone link. We demonstrate that the solution generates a very low number of false positives. We show how it detects some main attacks (including Mirai) arisen the last three years that traditional anomaly detectors have not seen. Details about noticed anomalies are provided to help the administrator qualifying them through specific features.