Intelligent Feedback and Evaluation for the Assessment and Improvement of Student Peer Reviews

Research into peer evaluation shows positive effects on student achievement and attitudes. To foster authentic peer reviews, students need to know their peer reviews will be seen and graded. However, the time-consuming nature of assessing feedback makes it impractical for instructors to evaluate thousands of feedback items generated by students in massive online courses. This paper describes the development of a deep learning (DL) tool to help assess the quality of feedback written by students in a peer-review system. We report tool development progress built upon years of student meta-feedback data that consists of numeric scores and written evaluations of the feedback received. We describe: the underlying DL platform, the labeled data set with which the algorithm was trained, the supervised training process, and the performance evaluation of the resulting three-class multiclass classifier. The evaluation showed satisfactory reliability when compared to student meta-feedback evaluations. We conclude by identifying issues to be addressed and the applications for this tool in massive online courses, including but not limited to giving formative feedback to students as they evaluate peers and helping identify underperforming reviewers.