Eliminating Unanswered Questions from Question Answering System for Khulafaa Al-Rashidin History

This paper discusses some attempts conducted to improve performance of Question Answering System for Khulafaa Al-Rashidin (Caliphs) history (called as QAKH). Experiments done on QAKH in 2012 showed that only 61,67% questions got a correct answer. The one contributed in this achievement was lack of Indonesian stemming process that implemented by utilizing Lucene library. It was found that Lucene did over stemming on some Indonesian word in indexing phase, so that it was delivered impacts on passage retrieval and answer extraction as well. We tried to implement two approaches in order to solve that problem. As the first, we used other library for doing stemming process for Indonesian words that developed by Information Retrieval laboratorium in University of Indonesia. In any case if there is no answer delivered, we shifted to the second approach where there is no stemming process applied. Result of evaluation demonstrated a better achievement where we got 66,67% correct answers. Analysis of each fold of experiments also discussed.