Put the Students to Work: Generating Questions with Constructive Feedback

Feedback has been long understood to be a vital component of learning. This is even more important in an online environment, where access to teachers is limited or unavailable. When online learning material is interspersed with opportunities for students to test their knowledge, there is a golden opportunity to give feedback and fix misconceptions immediately. All too often this opportunity is missed or mishandled in online learning platforms - either providing no feedback, or providing it at a coarse level of granularity. Furthermore, generating constructive feedback at a fine level of granularity is so time consuming as to be prohibitive. In this work, we build upon literature that has established students can be recruited to generate quality multiple choice questions, by exploring if students can also generate quality feedback to compliment the questions they generate. In the first iteration of an introductory programming course, we tasked 35 students to generate six questions each over three weeks, and analysed this dataset to generate principles of creating good multiple choice questions with constructive feedback. In the second iteration of the course, we repeated the same task, but provided students with the principles. The results showed an increased occurrence of good feedback when compared to the previous iteration with minimal need for additional intervention by teachers.