Comparing passive network monitoring of grid application traffic with active probes

Distributed applications require timely network measurements so that they can adapt to changing network conditions and make efficient use of grid resources. One of the key issues in obtaining network measurements is the intrusiveness of the measurements themselves - how much network performance is "wasted" taking the measurements? Our goal is to combine active and passive monitoring techniques to reduce the need for intrusive measurements without sacrificing the accuracy of the measurements. We are developing a bandwidth monitoring tool as part of the Wren network measurement system that will reduce the burden on the network by passively obtaining measurements from existing application traffic whenever possible, instead of actively probing the network. By using passive measurements when an application is running and active measurements when none are running, we can offer accurate, timely available bandwidth measurements while limiting the invasiveness of active probes. We have completed a prototype of the Wren bandwidth monitoring tool and present our preliminary analysis of its performance. We provide results from passive implementations of several available bandwidth techniques and demonstrate the close quantitative relationship between the results of both active and passive techniques. We have tested our implementation in a cluster, across a campus, and across the Internet using bulk data transfers as well as an adaptive eigenvalue application. Our results with this diverse set of environments and traffic types show promise toward implementing these techniques as measurement services in production environments.