Scalability analysis of semantics based distributed document clustering algorithms

Document clustering is an unsupervised learning paradigm. It provides efficient representation and visualization of the documents; thus helps in easy navigation. The importance of document clustering emerges from the massive volume of textual documents created. Here, semantics based distributed document clustering using Hadoop and MapReduce is proposed. Distributed version of two well-known algorithms, K-Means and Bisecting K-Means, is implemented; testing and comparison of both these algorithms is done for scalability and stability on single-node and multi-node Hadoop cluster. Also, scalability of the cluster setup and both the distributed algorithms is verified. The scalability of the algorithms is based on the Speedup, Scaleup, and Sizeup parameters.