All pair shortest path using distributed architectures

Due to the explosion of data from various sources, data analytics is found to be difficult using the CPU alone. For huge networks, the most popular graph algorithms using a single processor failed to accomplish this task. Hence the need of algorithms that have higher processing capabilities became dominant. Graph analytics is gaining importance in the realm of data analysis due to the advantages over other conventional analytical methods. All pair shortest path problem or path-finding problem has applications in various fields. Finding the all pair shortest path in a graph is computationally complex and time consuming in the case of large networks. In this paper, we propose a map reduce approach using the in-memory computation for finding the all pair shortest path using the transitive closure property and the greedy technique in Dijkstra's single source shortest path method. In order to overcome the scalability issues in the network representation, we use a tuple based network representation. The performance of the proposed method is proved experimentally.

I. Introduction

There are many ways of representing huge data and one such ways of representation is in the form of graphs. The advantage of modeling the data in the form of a graph is that it helps relating the data items in the form of relationships. In most of the cases they can be linked directly and can be retrieved with a single operation. Graph modeled data helps in dealing with various problems and has various applications as well. Path finding is an important class of problem in graphs.