Capturing Connectivity Graphs of a Large-Scale P2P Overlay Network

Measuring accurate graph snapshots of peer-to-peer (P2P) overlay networks is essential to understand these systems. Furthermore, the captured graph snapshots can be used, among other important purposes, as traces for simulation studies, to validate existing simulation models, to design and implement targeted attacks, or to detect anomalies. Motivated by the importance of the purposes above as well as the popularity of several Kademlia-like networks, we present a new crawler aiming to capture snapshots of the connectivity graph of the entire KAD network. The crawler's design is generic and adaptable for Kademlia-like and other structured P2P networks. The results show that the crawler is fast and captures high accurate graph snapshots. Furthermore, its design enables it to outperform prior KAD crawlers significantly in terms of the time and the number of crawling messages that are required to download nodes' routing tables. The crawls that we conducted at different times between April 2012 and February 2013 show that KAD is still widely-used in terms of total observed users. However, when compared to the results of prior studies, we report a significant drop in the number of its simultaneous online users.