Automated Student Code Scoring by Analyzing Grammatical and Semantic Information of Code

An automated code scoring system for introductory programming questions helps teachers keep track of students’ status and improve students’ learning efficiency. The most widespread current code scoring systems are based on dynamic testing (i.e., the number of passed test cases). However, this method does not provide insight into students’ programming abilities. In this paper, we aim to automatically score students’ codes base on classifier after extracting grammatical and semantic information. In particular, we propose an automatic label method based on code token distance, which can alleviate the bias problem of the manual scoring. Then we transform the code to the token sequence and the Structure-Based Traversal (SBT) sequence, which can capture grammatical and semantic information of the code respectively. To support our proposed method, we developed the Beijing University of Chemical Technology OJ (BUCTOJ) data collection system, which can collect data from students’ programming process. Then we conducted experiments on our gathered dataset. The experimental results show that our method can obtain promising results (i.e., 0.32, 0.04 and 0.92 in terms of MAE, Bias and Pearson correlation coefficient (r) performance metrics, respectively). Moreover, our proposed method can significantly outperform the method proposed by Singh et al. in a cross-question score prediction scenario.