An Investigation Into the Use of Large Language Models for Adaptive Grading of Course Projects

Grading in educational assessments via human evaluators often faces challenges from variability and bias, especially in subjective areas, posing obstacles to fairness and reliability. However, the rise of Artificial Intelligence (AI) technologies, particularly Large Language Models (LLMs) - AI systems using transformer technology, with billions of parameters and trained on large text datasets - offers innovative solutions to these challenges by enhancing rubric-based grading as it can produce content that matches or exceeds human quality. This study explores the potential of LLMs to complement human judgment in evaluating graduate-level statistics projects, addressing a critical gap in research on AI applications in statistics education. Using a comparative methodology, the study examines grading outcomes from human instructors and LLMs to evaluate consistency and accuracy. Key findings revealed a moderate to strong alignment between ChatGPT and human evaluators, with an ICC of 0.72 in objective areas, and 0.65 in more subjective sections. Integrating LLMs like ChatGPT enhances grading reliability, reduces bias, and provides students with detailed, constructive feedback. These results highlight the significance of adopting a hybrid AI-human framework to develop more equitable and consistent grading practices, ultimately advancing the effectiveness of assessments in higher education.