Optimizing Path Tracing Probe Collection for Kafka-Based Storage: Identifying and Addressing Performance Bottlenecks

Efficient and accurate processing of Path Tracing probes is essential for using the Path Tracing technology and therefore analyzing and diagnosing service provider networks. This paper presents a modular and optimized probe collector that addresses several performance bottlenecks. The probe collector consists of three main components: collecting, processing, and publishing. The collector component captures the incoming probes from the physical interface, while the processing component inspects and formats the captured packets. Finally, the publisher component ensures efficient transmission of the processed packet to the designated Kafka topic. To evaluate the collector's performance under various traffic loads, a comprehensive testing methodology emphasizing packet loss rate, memory analysis, and CPU utilization is introduced. The results demonstrate the collector's ability to handle incoming Path Tracing probes with high rates and minimal packet loss. Different solutions for specific bottlenecks are described, which will help other developers to address their software problems. The resulting Path Tracing probe collector provides a reliable and scalable solution to collect, process, and publish the probes to Kafka, enabling further development in this area.