User Behavior Prediction using A Novel Sentence N-Gram Model

Humans normally use language contextually and make their decisions. This intelligence if transferred to a computer can assist in effective decision making. There is a need to replicate the contextual behavior of humans using web mining techniques. The aim of this work is to ingest real time data from social media data sources and harvest in a data warehouse. The raw data is preprocessed to eliminate the anomalies present raw data. The data is then parsed for creating user profiles. The N-Grams vocabulary is constructed and mapped with the synonym and abbreviation vocabularies. The user behavior is analyzed contextually using the user comments and their corresponding action in the past. The model is trained using the historical data extracted from the same sources. Various supervised machine learning techniques are used for predicting the succeeding comment and their performance is assessed. Conditional probability is assigned to each N-gram using the frequency of occurrence. The comment that has the highest probability value is predicted as the next comment. The performance analysis demonstrates that the prediction using ensemble approaches yield better results.