Accurateness evaluation of an IR-based marking engine for mathematics assessment

This paper presents the results of the preliminary evaluation of the accurateness of the automated marking implemented by an IR-based marking engine on free-response mathematics solutions. The development of the marking engine is based on adaptation of techniques in text retrieval process. The engine incorporates our own similarity measure which models a mathematical expression as a multiset of mathematical tokens. The measure evaluates the similarity between the mathematical expression of a worked-out solution and that of a model solution. The degree of correctness of a worked-out solution that has been compared is indicated by the similarity value. The results of the preliminary evaluation show that the IR-based marking engine is able to perform an automated marking of the solutions of the four (4) questions tested with an average of 97% acurateness when compared to human marking.