High Performance DPI Engine Design for Network Traffic Classification, Metadata Extraction and Data Visualization

The increasing volume and complexity of network traffic pose a significant challenge in today’s interconnected world. Internet service providers (ISPs) and telecommunication companies rely on traffic monitoring using Deep Packet Inspection (DPI) frameworks to monitor network traffic, including the detection of anomalous behavior. This paper presents the development of a high-speed and scalable DPI engine. This engine is designed to capture packets at high speeds, extracting data and metadata from these packets, ultimately providing profound insights into internet traffic patterns. The developed DPI engine comprises essential components, such as packet capture, time stamping, flow processing, traffic classification, and semantic analysis. It exhibits the capacity to process data at remarkable speeds, reaching multiple 10 GB/s, which is crucial in this era of rapidly evolving network traffic. Moreover, the paper demonstrates the creation of interactive dashboards that enable effective visualization of the extracted information, making it readily interpretable for users. Perhaps most significantly, the DPI engine is remarkably lightweight and easily deployable on standard hardware, requiring no specialized equipment. Its versatility allows ISPs to employ it for a variety of purposes, including traffic shaping, traffic charging, and ensuring high-quality service (QOS) for their customers.