Exploring the Capabilities and Limitations of Generative AI in Providing Feedback on Engineering Drawings: A Case Study

This paper explores and evaluates the use of OpenAI’s ChatGPT-4o as a feedback tool for engineering drawings. The AI feedback tool was trained on a synthetic dataset specifically created for the task, which included a comprehensive checklist derived from the Australian Standard for Technical Drawings (AS 1100) and a variety of engineering drawing examples. The evaluation involved repeated testing of three distinct case studies, each representing different levels of drawing accuracy. The findings reveal both strengths and weaknesses in the AI feedback tool’s performance, providing valuable insights into the capabilities and limitations of AI in providing feedback on engineering drawings in the education context. The research also highlights the need for further training to improve accuracy in certain areas and the potential bias in the evaluation process. A key observation from this study is that while Open AI’s ChatGPT-4o model can generate feedback, it may not truly ‘understand’ the engineering drawings in the same way a human would. This work contributes to the ongoing efforts to leverage AI in education, particularly in the context of engineering drawing feedback, and underscores the importance of human oversight and intervention in the application of AI technology.