Per-Flow Packet Loss Measurement without Probing

This paper addresses the fundamental problem of per-flow loss measurement: for any flow passing through two observation points, estimate the number of packets that leave the first observation point but do not arrive at the second. To the best of our knowledge, there is no prior work on per-flow loss measurement. In this paper, we propose the first per-flow packet loss measurement scheme called FLOE. Given a set of observation points, FLOE records information about size of each flow at each point so that later it can accurately estimate the number of lost packets of any flow between any two points. The key idea is to record information about the size of each flow at a set of observation points, during which noise is carefully introduced so that storage space can be minimized and the accurate loss estimate can be obtained later for any flow between any two observation points by statistically denoising the recorded information. FLOE advances the state-of-the-art on per-flow loss measurement from the following three major fronts: measurement granularity (as it measures per-flow losses), traffic interference (as it uses no probe packets), and accuracy guarantee (as it is guaranteed to satisfy the required reliability and confidence interval specified by operators). FLOE is designed to be efficiently implementable on network middleboxes. In terms of processing overhead, FLOE performs only one hash and one memory update per packet. In terms of storage space, FLOE uses less than 0.05 bits per packet, which means that, on a backbone link with about half a million packets per second, using a 256GB drive, FLOE can accumulate packet loss information of flows traversing the link for about 3 years. We evaluated FLOE using three real traffic traces that include a backbone traffic trace, an enterprise network traffic trace, and a data center traffic trace. Our results show that FLOE always achieves the required reliability for any given confidence interval and network topology.