A Graph Analysis Method to Improve Peer Grading Accuracy for Blended Teaching Courses

Peer grading is a tool widely used by instructors to provide fast reviews for homework that consists of open-ended questions. As the grades obtained from peer grading are not as accurate as those provided by the instructors, many methods have been proposed to improve the accuracy of peer grading. However, the current methods mainly focus on the scenario of online teaching, and they lose effectiveness in blended teaching courses because the mandatory of task and affinity among students may make the students perform irresponsibly in the grading task. This paper proposes a method based on graph analysis to improve the accuracy of peer grading. The peer grading system is modeled as a bipartite graph. In the graph, three interdependent metrics are defined to measure the dutifulness of the grader, the reliability of the rating and the true score of the submission. The stable values of the metrics are computed in an iterative way to obtain the peer grading results. Experiments demonstrate the proposed method is effective in blended teaching settings, and outperforms the current methods. Compared to the baseline of the mean value method, the proposed method decreases the root mean square error by 2.31% in the worst case and 30.72% in the best case on real-world data. It is robust to irresponsible graders, where the root mean square error keeps small even when the proportion of irresponsible graders increases to 30%.