Structured superpeers: leveraging heterogeneity to provide constant-time lookup

Peer-to-peer (P2P) systems are typically divided into those that centralize lookup functionality in a single location and those that distribute the lookup operation across the set of participating hosts. The former approach can offer constant time lookup latency, but is more expensive to scale and suffers from single points of failure. In contrast, the fully distributed approach is easier to scale and can be more resilient to failures, but the lookup latency scales as a function of the total number of participants. While the research community has made great progress in improving the latency of distributed lookup, these systems, exemplified by Chord[I. Stoica et al., (2001)] typically require O(logN) hops to locate an object in a system with N hosts. We explore the costs and benefits of a new hybrid approach that partially distributes lookup information among a dynamically adjusted set of high-capacity "superpeers". This design exploits the resource heterogeneity inherent in existing P2P systems to provide many of the advantages of a centralized system, even while avoiding most of the problems associated with such systems. Lookup is performed using superpeers in constant-time, and the system performs well even in the event of simultaneous super-peer failures. Finally, while our gain in performance is potentially at the expense of scalability, we will show that a straightforward implementation should be able to scale to over one million peers with reasonable lookup rates.