Local and Global Data Spread Based Index for Determining Number of Clusters in a Dataset

Most of the clustering algorithms are sensitive to the input parameters and produce different clustering results for different input parameters for same datasets. A number of methods and indices have been proposed for validating results of a clustering process. The most commonly used approaches for cluster validation are based on internal indices. In this paper, we propose a new cluster validity index (ARSpread index) for the purpose of cluster validation and determining number of clusters present in a dataset. Local and global data spread based approach is proposed to measure the compactness of a cluster. A distinctness measure that is based on a penalty function is incorporated in the proposed index. We conduct a thorough comparison of five commonly known indices with the proposed index and provide a summary of experimental performance of different indices. Experimental results show that the proposed new index performs better than the commonly known indices.