RaNT-Graph: A Scalable Approach to Sampling Billions of Walks or Paths from Weighted Graphs

Sampling random walks is a crucial component of many graph algorithms that perform graph embedding, link prediction, and other tasks. The effectiveness of these stochastic algorithms coupled with the emergence of large graphs across various domains underlines the importance of sampling walks at scale. Most approaches target shared memory systems where compute and memory resources limit scalability. We extend our previous work on Random Neighbor Traversal Graph (RaNT-Graph), a distributed data structure initially designed to scale the sampling of paths. RaNT-Graph’s use of vertex delegation partitioning and asynchronous walk messages provide the ability to scale not only the sampling of paths but also walks. We develop a novel algorithm for constructing distributed alias tables, and integrate it with RaNT-Graph to enable sampling paths or walks from weighted graphs. In addition, this work presents the concept of a state manager, which reduces the communication cost of particular walk sampling tasks. When comparing RaNT-Graph to the state-of-the-art distributed walk sampling engine, it provides a 3.62x speedup on average. Furthermore, we exhibit RaNT-Graph’s adeptness for massive workloads by sampling 3.4 billion walks from a graph possessing 224.5 billion edges in under 18.2 seconds using 256 compute nodes (14,336 cores).