Fast Anomaly Detection for Large Data Centers

Recent spates of cyber attacks towards cloud computing services running in large data centers have made it imperative to develop effective techniques to detect anomalous behaviors in the "clouds". In this paper, we propose to use the distributions of IP address octets and centroid based measures to characterize the inherent IP structure in high-volume data center traffic, and subsequently design a simple yet effective algorithm to detect abnormal traffic patterns caused by network attacks such as worms, virus, and denial of service attacks. We evaluate the effectiveness and efficiency of this algorithm with synthetic traffic that combines real data center traffic collected from a large Internet content provider with worm traces and denial of service attacks. The experiment results show that our algorithm consistently diagnoses the abnormal traffic from normal ones, and does so in a short time with a low false alarm rate. We believe that the proposed approach could be potentially deployed in real-time data center environments to enhance the security and high availability of cloud computing.