A Framework for Revision of MOOC Formative Assessments

Formative assessments are critical components of any learning experience, and in our Massive Open Online Courses (MOOCs), we apply the learning sciences to design problem-solving activities. In our late-stage design process, we analyze data from, revise, and iterate on our formative assessments after initial creation and implementation. There is a lack of easy-to-use yet data-driven guidance for instructors to go through these steps systematically to evaluate their online course assessments. Using learner performance data and feedback, we establish a framework to evaluate instructor- and learner-based causes of incorrect responses to computer-graded questions. We use this framework to streamline the process of formative assessment revision to maximize the effectiveness of learning. Course designers and instructors can apply this revision approach to online formative assessments in hybrid and fully online settings across a range of course sizes or platforms.