Impact of noise in distance-based short text clustering

Efficient text clustering is essential for applications like recommendation systems, trend analysis, and user profiling, particularly when dealing with data from online repositories such as social media platforms. However, the effectiveness of clustering methods is significantly impacted by the inherent challenges of such data, including sparsity, short text length, and unstructured nature. One of the most critical challenges is the prevalence of noise in the data, manifested through slang, emojis, misspellings, abbreviations, and grammatical errors, which can severely degrade clustering performance. This research examines the robustness of distance-based clustering algorithms against noisy data by systematically introducing noise directly into the text. We introduce two types of noise–attribute noise and class noise for effective noise injection. We analyze the ability of different distance measures to differentiate meaningful content from irrelevant information under these noise-induced conditions. Through a series of experiments, we evaluate the degradation in clustering accuracy of various distance measures in the presence of noise, providing valuable insights into their noise tolerance and practical utility in real-world scenarios.