Automation of assessment and feedback in IT teaching from the teaching staff perspective

Information technology (IT) has been a popular specialty among high-school graduates for quite some time. As the number of entrants continues to grow year-by-year, the load on teaching staff also increases. Automation of teaching activities can be used to alleviate the high workload resulting from courses with hundreds of students. The aim of this study is to ascertain the processes that could be automated in the context of assessment and feedback. To gather ideas, eight one-hour mini-group interviews were conducted at the Institute of Computer Science, the University of Tartu, with 17 faculty members responsible for teaching IT courses. A qualitative approach was used for classifying the proposed ideas related to the automated assessment and automated feedback. Five main categories connected to automated assessment emerged from analyzing the codes generated based on the interviews using qualitative content analysis: use of automated assessment, instructor-related topics, negative effect on students, improvements of automated assessment, and technical concerns. Regarding automated feedback, three main categories emerged: feedback for teachers, feedback for students, and feedback for both teachers and students. The classification of ideas was used as an input for designing a schema displaying the current automated assessment system with all the potential additions suggested by the faculty members showing how, if and where exactly the proposed ideas would fit. We believe that the ideas and improvements related to automated assessment and feedback in this study, in conjunction with the detailed clarification of the process, can be a useful input for other institutes. The results will be used as a basis for implementing and improving the automation tools used hand in hand with teaching by the faculty members.