Indonesian Stemmer for Ambiguous Word based on Context

Stemming is a technique to remove affixes from a word to obtain a stem. One problem in stemming Indonesian was the morphological ambiguous word, such as “memerah” (blush or milking) which can be stemmed to “merah” (red) or “perah“ (milk). Previous deterministic stemmers did not handle this ambiguity. Meanwhile, the nondeterministic approach succeeded in finding the word candidates, but it could not choose the correct stem due to the absence of context. In this research, the nondeterministic approach was improved by adding the context, namely words that precede and follow the ambiguous word. Like nondeterministic approach, the affix rule checker in the proposed stemmer generated several word candidates from the input word. Then, the candidates are selected using two steps. First, they filtered by root word dictionary, list of compound words, and list of rare words. Second, if there were still more than a word candidate, the best word was the one with the highest average similarity score from the word to its context and the ambiguous word itself. To calculate the similarity score of words, a word model derived from word2vec training was used. Experiment results showed that the proposed stemmer accuracy was 99.65% on average. Whereas the deterministic and nondeterministic stemmer reached a 98.36% and 99.32% average.