Statistical Evaluation of Control of Performing Works with Free Response in Engineering Education

In online learning, the main knowledge and skill assimilation control tool is tests. Also in learning management systems, such a tool as an assignment is used for control over performing works with free response (WWFRs) (laboratory works, course works, etc.). The application of this tool is particularly important in case of engineering education, when students perform a significant number of WWFRs throughout the entire period of study. The means of control statistical evaluation in training is necessary for the identification of problems with tests and test items, and also with course materials for their subsequent revision. The current solutions are primarily aimed at the statistical evaluation of tests and items belonging to them, at the same time there are used two approaches to pedagogical measurements: classical test theory (CTT) and item response theory (IRT). Several studies are focused on questions related to the assignment evaluation. However, in the former, the emphasis is not on the assignment itself (the correctness of elements it contains), but on other aspects mainly connected to the assessing procedure. In the paper, a new approach to statistical evaluation of control over performing WWFRs, considering the assignment as a system of its parts, utilizing typical mistake base and rooted in a dichotomous two-parameter (2PL) IRT-model, is proposed. The approach was piloted on data on performing a calculation and graphic work on the university course “Project Workshop” with the usage of author’s software.