A Metalearning Approach to Personalized Automatic Assessment of Rectilinear Sketches

Sketchtivity is a stylus-based intelligent tutoring system that can help instructors automatically provide feedback to their students, saving them the time and effort of providing personalized feedback themselves. The system uses a generic evaluation of perspective, direction, and accuracy to give students feedback on the quality of their sketches. If instructors want to personalize the metrics, the system would require them to provide multiple sets of samples. Therefore, instructors may use instructional team members such as teaching and graduate teaching assistants to provide feedback on the required samples. Compared to that of assistants, the feedback they produce might vary due to expertise and create noise in the training data. To address this problem, we implement a deep neural network that leverages learning to reweight algorithms. The data collected by the instructor from undergraduate and graduate-level rectilinear perspectives sketching is considered the validated sample. In this study, we analyzed the training size requirement for a Multi-Layer Perceptron (MLP) to accurately predict whether or not a stroke was a perspective stroke. We observed that the training data required to predict stroke accuracy is small. In addition, the performance of the algorithm in terms of accuracy was good even under extreme conditions such as having highly unbalanced data and having a small valid set of data. The results from the study support the use of these types of algorithms for future system personalizing to support scalable feedback systems in education.