Applying Faulty Statement Category Frequency to Localize Faults for Student Programs

Online Judge (OJ) system has been widely used in the teaching of programming courses. When students use the OJ system to submit programs, OJ can only provide simple feedback on whether the program is correct or not. However, students with little programming experience are difficult to find and fix even simple bugs. Although a lot of approaches were proposed to localize faults in open-source and industrial software programs, whether these techniques can achieve satisfactory results in student programs has not been thoroughly investigated. In this paper, we propose a Faulty Statement category Frequency-based Fault Localization (FSFFL) method to locate bugs in student programs. Specifically, motivated by the observation that the faulty statement categorization for the same OJ question is always similar, we employ the history information of student programs and divide the corresponding code statements into seven categories, and then we calculate the faulty frequency for each category. Finally, we use the frequency information to give weight to the suspiciousness values of statements calculated by traditional Spectrum-Based Fault Localization (SBFL) methods. To verify the effectiveness of our proposed approach, we conduct empirical studies on 2,411 faulty programs from the OJ platform. Experimental results show that our approach with the Dstar formula can improve the fault localization accuracy by 18.73% on average compared with the traditional SBFL methods in terms of EXAM metric.