Studying the performance of clustering techniques for biomedical data using spark

The size and growth of data from many sources has been brought us to the generation of big data era where the volume and amount of data cannot be computed and processed using conventional or traditional environment. To process and compute the big data there are lots of big data environments are developed such as Hadoop using Distributed File System (HDFS) and MapReduce framework. Apache Spark is the newly developed framework that can be used with hadoop and it is built and run on top of it. In this paper, we study the performance of different clustering techniques of data mining using spark. The clustering algorithms used are K-means, Bisecting K-means and Gaussian Mixture models. Our design uses Spark Resilient Distributed Datasets to store the colorectal cancer patient's data in HDFS. The results show that the algorithms yield a different set of clusters and a fact common to all algorithms is that the average survival months of a patient decreases as the cluster center increases.