Multilingual Text Categorization of Indo-Aryan Languages

This paper explores a generalized document categorization technique on both Bengali and Nepali documents. The motivation for the research is to develop a common Natural Language Processing (NLP) algorithm for the languages which are originated from the same language family tree. In this regard, several machine learning (ML) approaches; specifically Linear SVC, Chi-squared SVC, Bernoulli Naïve Bayes, Multinomial Naïve Bayes algorithms are used to automatically classify Bengali and Nepali documents into predefined categories. We analyze the efficiency of the algorithms and validate the hypothesis by showing more than 90% accuracy of all the algorithms. We also, observe interesting analogy among top features of Bengali and Nepali categories.