Predicting Guesses and Slips Through Question Encoding with Complexity Hints

One commonly researched adaptive learning method, knowledge tracing, supposes that a learner answering an exercise correctly does not immediately imply that they have mastered the associated knowledge component, and vice-versa for answering incorrectly. Alternative explanations to learner performance such as guesses and slips reduce the accuracy of assessment items in determining learning achievement. One possible contributor to the learner’s tendency to make guesses or commit slips is the unnecessary complexity of a problem. In this paper, the potential usefulness of complexity hints in predicting mastery of algebra problems is explored. Natural language processing methods were used to derive a complexity estimate of each assessment problem and applied to estimate guess and slip probabilities using individualized neural network models. An added important advantage of complexity estimation over previous methods for knowledge tracing is its potential contribution to assessment design by revealing which problems tend to be more confounding than helpful to learner progress. This potentially may lead to new lines of research in natural language processing applications, knowledge tracing, and curriculum quality assurance.