0 to 10k in 20 Seconds: Bootstrapping Large-Scale DHT Networks

A handful of proposals address the problem of bootstrapping a large DHT network from scratch, but they all forgo the standard DHT join protocols in favor of their own distributed algorithms that build routing tables directly. Motivating their algorithms, the proposals make a perfunctory claim that the standard join protocols are not designed to handle the huge number of concurrent join requests involved in such a bootstrapping scenario. Moreover, the proposals assume a pre-existing unstructured overlay as a starting point for their algorithms. We find the assumption somewhat unrealistic. We take a step back and reexamine the performance of the standard DHT join protocols. Starting with nothing other than a well-known bootstrap server, when faced with a large number of nodes joining nearly simultaneously, can the standard join algorithms form a stable DHT overlay? If so, how quickly? Our simulation results show that Chord and Kademlia's join protocols can actually handle the bootstrapping scenario quite well. For 10,000 nodes joining at a rate of 1,000 nodes per second, Chord and Kademlia took less than 20 and 15 seconds, respectively, to form a stable overlay. The Chord join protocol, however, requires a slight modification for fast bootstrapping. We elucidate the reason why the modification is necessary.