IDES: An Internet Distance Estimation Service for Large Networks

The responsiveness of networked applications is limited by communications delays, making network distance an important parameter in optimizing the choice of communications peers. Since accurate global snapshots are difficult and expensive to gather and maintain, it is desirable to use sampling techniques in the Internet to predict unknown network distances from a set of partially observed measurements. This paper makes three contributions. First, we present a model for representing and predicting distances in large-scale networks by matrix factorization which can model suboptimal and asymmetric routing policies, an improvement on previous approaches. Second, we describe two algorithms-singular value decomposition and non-negative matrix factorization-for representing a matrix of network distances as the product of two smaller matrices. Third, based on our model and algorithms, we have designed and implemented a scalable system-Internet Distance Estimation Service (IDES)-that predicts large numbers of network distances from limited samples of Internet measurements. Extensive simulations on real-world data sets show that IDES leads to more accurate, efficient and robust predictions of latencies in large-scale networks than existing approaches