Modeling a Hybrid Stemmer for Kokborok

Kokborok is a resource scarce and vulnerable language and is spoken only by around a million people in the north-east Indian state of Tripura. Lots of unstructured textual data in Kokborok is now available however efficient data mining for such a resource is not applicable. Reasons are the absence of an efficient pre-processing step that can help in solving the information retrieval or natural language processing. In this work, we present a model of hybrid stemming algorithm for the vulnerable and resource-scarce Kokborok language and assess its performance using various methods. Evaluation of the hybrid stemmer was performed on a corpus of 52,482 words that resulted in achieving an accuracy of 89.28%. Recall and Precision of 89.28% and 89.61 %, respectively, with F1-measure of 89.44 has been found for the stemmer.