Optimizing Large Language Models for Auto-Generation of Programming Quizzes

This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of using LLMs in CS education from educators' viewpoints, and identify effective prompt engineering strategies to enhance the quality of educational materials. The research compares quizzes made by LLMs against human-created content to assess their consistency with Java programming principles, alignment with CS1 and CS2 learning goals, and their impact on student engagement and comprehension, providing insights into LLMs' effectiveness in academic assessment creation for computer science education.