Measuring and visualizing learning with Markov Models

In this paper we introduce new tools for measuring and visualizing the learning process of students in large classes where students are required to use a computer-supported problem solving learning environment (PSLE) either in class or as homework. These learning environments give students unlimited opportunities to propose a solution to the problem and give them immediate feedback. We show how to create a graphical representation, the Problem Solving Markov Model (PSMM), of the set of all attempted solutions proposed by the class. Each node in the PSMM consists of an equivalence class of attempted solutions and each edge corresponds to a transition from one attempted solution to another in one step. The edges are typically labeled with the number of times a students went directly from one attempted solution to the next, or with the probability of going between the two nodes. We give examples from two tools: CalcTutor for Calculus problems, and Spinoza for Java or Python programming, and we provide several pedagogical applications of PSMMs which give the instructor a simple way to obtain a highly nuanced understanding of mastery of the problem solving process for individual students and for the class as a whole.