Trend analysis in online data with unsupervised classification and appraisal categories

This paper presents a hybrid approach for identifying trends in social media datasets. This approach uses jointly unsupervised methods for text classification and an ontology of appraisal categories. First, data gleaned on social media are classified with unsupervised methods in order to produce more or less homogeneous clusters. Then, topics are detected within each cluster by using Latent Dirichlet Allocation (LDA). At cluster level, appraisal categories are identified thanks to an ontology build according to the principles of the Appraisal Theory. The joint identification of topics and appraisals offers a basis to analyze trending topics in social data. The paper investigates a novel means to detect trending topics in social data by utilizing unsupervised classification methods and focusing on subjective states such as affect, attitude, denial, disapproval, rejection, endorsement or support, associated with each class. Negative or positive polarity and intensity degrees are also identified, thanks to the appraisal ontology. The approach identifies the most dominant trends in social data as associations of topics and appraisal categories. The paper also discusses experiments carried out to detect trends on Twitter collections and the evaluation of theirs results in the light of manually validated data.