Skampling for the Flow Duration Distribution

This paper concerns the problem of estimating the Internet flow duration distribution from indirect measurements due to network constraints. The aim is to estimate the distribution from observing: the possible superpositions (collisions) of sampled flow durations, the flow arrivals-to-departures times without identification of sampled flows and the number of sampled flows in progress. For each type of data available, we present estimators of the flow duration distribution, formulating the problem in queueing system terms. We also propose data streaming algorithms using sampling and sketching (through counters) to obtain the considered partial information from flows. At the core of this skampling (i.e. sampling and sketching) approach is the ability to tune the flow sampling probability for "optimal" flow load onto sketch entries (queues). Finally, we present numerical results comparing the different estimators of the flow duration distribution using two real Internet traces.