Measuring and analyzing service levels: a scalable passive approach

Internet service providers are increasingly trying to differentiate themselves in terms of the service performance that they provide to their users. In this paper, we have developed a scalable service level monitoring methodology to assess user satisfaction without injecting any measurement traffic. Specifically, we suggest Web throughput as a service level metric, outline possible ways to measure it and discuss the advantages of passive observations of actual user activity. We further propose a statistical data analysis method that analyzes passive throughput measurements and quantifies user satisfaction/dissatisfaction and the confidence that the provider may have in the collected data, i.e. data reliability. The proposed technique is based on the premise that the service provider is interested in continuously monitoring the service levels being offered to a majority of the users over a long enough time. We present results of a real-world experiment that demonstrates that, with careful data analysis, passive measurements can effectively detect service problems. Our experiments also indicate that, for 90% of the time, the results of reliable passive measurements agree with those of random active measurements. Unlike active measurements, passive measurements do not generate additional traffic in the network, and hence are preferred. The underlying approach may also provide a communication vehicle between service sales/marketing and operations/capacity planning aspects of service provisioning.