Leveraging LLMs for Student Assessment Generation in Engineering Education: a Study on Context Requirements and Adoption

Instructors are leveraging LLMs in various innovative ways to enhance assessment generation and overall educational practices within engineering education. By focusing on practical engagement, personalized feedback, and the development of new assessment methodologies, educators aim to create a more dynamic and effective learning environment. The current work systematically examines the capabilities of Large Language Models (LLMs), such as GPT, Claude, and Copilot, to augment pedagogical practices and learning outcomes in university environments, with a particular emphasis on their use in quiz generation. Specifically, three difficulty levels were employed-elementary, intermediate, and advanced-along with three question formats: multiple choice questions with one correct answer, multiple choice questions with more correct answers, and open-answer questions. The study involved a sample of five instructors from two engineering departments, along with 24 students, to assess their perceptions and experiences within the designed context. A Technology Assessment Model (TAM) was employed to evaluate the adoption and integration of LLMs within educational contexts. The findings reveal that both educators and students regarded LLMs as intuitive and user-friendly, demonstrating a strong conviction in their potential to enhance instructional activities, particularly assessment tools. Index Terms—AI, Large Language Models, GPT, CoPilot, Claude, Engineering Disciplines, Technology Acceptance Model.