Automatic Grading of Free-Form Diagrams with Label Hypernymy

The automatic grading of free-form answers is gaining importance. This paper presents the only system we know of capable of automatically grading free-form diagrammatic answers. Our approach identifies the parts of a diagram that carry specific meaning in that domain, termed minimal meaningful units (MMUs), and matches them with equivalent parts in a model solution. The result of the matching is used to calculate the grade and to generate appropriate feedback. The matching is complicated by errors, omissions, and superfluous items in the student answer. In particular, inheritance hierarchies in diagrams can cause labels in the diagram to be fragmented across several objects. Our approach has been successfully applied to a variety of diagram types and grades diagrams better than a human marker. We briefly describe a set of tools that allow the easy creation of questions, marking schemes, and diagram editors suitable for online assessment and embedding in a VLE quiz engine.