HashCat: A Novel Approach for the Topic Classification of Multilingual Twitter Trends

With the growing usage of online social networks, an enormous amount of data is generated by users daily. Twitter microblog groups tweets of the same hashtag which is beneficial for the users to extract the required information for the target hashtag effortlessly. However, understanding these hashtags is a challenging task as tweets contain short, multi-lingual content and non-standard vocabulary. In this article, we propose HashCat – a novel approach for the topic classification of multilingual Twitter trends. In addition, we present a technique for the segmentation of English and Urdu language hashtags. First, we develop a labelled dataset of HT-Dat containing 1,882 hashtags of Urdu and English languages by manually labelling them into six wide range categories. Next, we utilize the features of i) tweet text, ii) co-occurrence and iii) segment similarity for the classification of hashtags. The HashCat achieves an overall accuracy of 0.93 on the HT-Dat dataset. The classification results and Type-Token Ratio analysis for various hashtag categories reveal that the categories with low lexical diversity are classified with higher accuracy by the HashCat classifier. We believe that our methodology can be helpful for social media analysts to conduct research on specific domain hashtags.