Dealing with Bootstrapping, Maintenance, and Network Partitions and Mergers in Structured Overlay Networks

In the last decade, numerous structured overlay networks were proposed as a scalable infrastructure to build large-scale distributed systems under dynamic environments. These overlays were touted to be fault-tolerant and self-managing, yet, as we show in this paper, they fall short of handling some extreme scenarios they envision. These scenarios include bootstrapping, and underlying network partitions and mergers. We argue that handling such extreme scenarios is fundamental to providing a fault-tolerant and self-managing system, and thus, structured overlay networks should intrinsically be able to handle them. In this paper, we present ReCircle, an overlay algorithm that apart from performing periodic maintenance to handle churn like any other overlay, can merge multiple structured overlay networks. We show how such an algorithm can be used for decentralized bootstrapping. ReCircle does not have any extra cost during normal maintenance compared to an isolated overlay maintenance algorithm. Furthermore, the algorithm is tunable to tradeoff between bandwidth consumption and time to convergence during extreme events like bootstrapping and handling network partitions and mergers. We evaluate the algorithm extensively under various scenarios through simulation and experimentation on Planet Lab.