Probabilistic parallel measurement of network traffic at multiple locations

Measuring the per-flow traffic in large networks is very challenging due to the high performance requirements. In addition to that, if traffic is measured at multiple points in the network at the same time, it becomes necessary to merge the observations in order to obtain network-wide statistics. When doing so, packets must be accounted for only once, even if they traverse more than one measurement point. Today's standard technique, sampling-based traffic accounting, results in large approximation errors. Here, we describe an approach named Distributed Probabilistic Counting (DPC). DPC is based on a probabilistic data representation. It provides accurate traffic statistics at very low per-packet effort, and is able to merge measurement from multiple network locations while counting each distinct packet only once.