Tweets clustering: Adaptive PSO

In the today's world, huge amount of online data is required for various analysis purposes. It is difficult to store, manage and retrieve such a massive data efficiently, especially when the data is continuously getting appended during run-time. This generates the need to organize such a data in similar groups in a more dynamic fashion. Not being adaptive in nature, traditional algorithms like K-means fail to accommodate the newly arrived data during run-time, without re-initialization. In this approach, we have carried out clustering of streaming tweets using adaptive particle swarm optimization (APSO) algorithm. The algorithm is adaptive, because it can accommodate the streaming data effectively and efficiently, without having to go for re-initialization. Unlike previous approaches, we have initialized the particles, only at the commencement of the algorithm, in such a way that they are well-distributed and thus cover the complete problem space leading to the algorithm not getting stuck at the local optimum. We have also devised a mutation-like operation, which at the arrival of new tweets; re-initialize only a subset of the converged particles away from the convergence point. This accommodates the latest data effectively, again covering the complete problem space. With this approach, we have achieved a trade-off between the cluster quality and the execution time.