INTCollector: A High-performance Collector for In-band Network Telemetry

In Software-Defined Networking (SDN), monitoring is an essential function to provide information about the network and help the SDN controller to make network controlling decisions. In-band Network Telemetry (INT) is a method that can provide real-time, fine-grained, and end-to-end network monitoring. INT works by embedding network information into every packet. At the final switch in the path, INT extracts network information into report packets and sends the reports to a collector. However, the huge amount of data from INT requires high processing capability of the collector.We present the design and implementation of INTCollector, a high performance collector for INT. We propose a mechanism to extract important network information, called event, from INT raw data. The mechanism filters network events, reduces the number of metric values that need to be stored, reduces CPU usage and storage cost while still ensuring to capture and store all important network information. INTCollector has two processing flows: a fast path to process INT report packets, and a normal path to process events and store metric values into a database. The fast path is accelerated by eXpress Data Path (XDP) -a Linux in-kernel fast packet processing framework. Our calculation shows that event detection can massively reduce the amount of data need to be stored (two to three orders of magnitude in our test scenario). The evaluation shows that INTCollector can process INT reports at the rate of 1.2 Mpps with 8% of CPU when running with software NIC. We expect better result can be achieved with XDP supported hardware NICs.