A framework for near real-time event characterization within the internet

The Internet occasionally experiences large disruptions arising from both natural and manmade disturbances. It is of significant interest to develop methods to characterize these events. The characterization comes in the form of detecting disturbances early in their evolution, providing a classification of disturbances, and locating disturbances within the topology of the Internet. This paper presents a new framework for realizing near real-time global scale disruptive Internet event detection, classification, and localization. The proposed framework uses Hidden Markov Models for event detection and classification, and tensor decomposition and graph theoretical analysis for logical localization. The framework consists of three major components: 1) data ingest and processing/cleaning, 2) near real-time alert generation, and 3) a human-in-the-loop analysis portal. This paper discusses components 1 and 2 in detail. Components 1 and 2's usefulness, as well as an overview of component 3, is demonstrated through the use of case studies. This framework outlines a principled, automated approach to characterizing network disruptions at Internet scale.