A Comparative Evaluation of Big Data Frameworks for Graph Processing

A lot of big data applications are using the graph model for structuring data. Examples are digital social networks, traffic information systems, and the hypertext connection between web pages. In order to process large scale graph structures, various frameworks have been proposed. In this paper, we evaluate and compare GraphX based on Spark and Gelly based on Flink as two prominent graph processing frameworks. We focus on the scalability of GraphX and Gelly with respect to increasing data volumes and their ability to distribute work between multiple processing nodes in a cluster. We perform experiments with different graph algorithms and both real world data and artificially generated data. For this, we implement a new algorithm using both Flink and Gelly. Our experiments show that while both frameworks are capable of processing big data, they substantially differ in non-functional features like resource consumption. Additionally, our experimental data show that choosing between different computing models offered by the frameworks can significantly influence the performance of big data graph computations.