Scaling Semantic Graph Databases in Size and Performance

This article presents SGEM, a full software system for accelerating large-scale graph databases on commodity clusters. Unlike current approaches, GEMS addresses graph databases by primarily employing graph-based methods, which is reflected at all levels of the stack. On the one hand, this allows exploiting the space efficiency of graph data structures and the inherent parallelism of some graph algorithms. These features adapt well to the increasing system memory and core counts of modern commodity clusters. On the other hand, these systems are optimized for regular computation and batched data transfers, whereas graph-based methods usually are irregular and generate fine-grained data accesses with poor spatial and temporal locality. The framework comprises a SPARQL-to-C++ compiler; a library of distributed data structures; and a custom, multithreaded, runtime system. The authors introduce their stack, discuss its advantages with respect to other solutions, and show how they overcame the challenges posed by irregular behaviors. They evaluated their software stack on the Berlin SPARQL benchmark with datasets of up to 10 billion graph edges, demonstrating scaling in dataset size and in performance as they added nodes to the cluster.