Recognizing Patterns in Text Data through Effective Initialization of Spherical K-means

Spherical k-means is a fast and effective method for clustering text documents. But its performance is greatly affected by the initial centroids or seeds. It has been established that D2-sampling based methods like kmeans++ are very effective initialization methods for k-means to produce well separated seeds. A similar adaptation for spherical k-means is not possible due to huge dimensionality of text corpus. Pairwise distance among all text documents is impractical. We propose two seeding methods for spherical k-means that performs equally well but are computationally affordable. Separation among output clusters and cohesiveness are used to evaluate performance of the initialization methods over popular text corpus.