EyeUnderstand: Dashboard for Gaze and Deep-Learning Driven Comprehension Estimation in Online Lectures

Online videos are a potent tool for educators to disseminate knowledge widely to diverse student audiences. However, collecting student feedback remains a significant challenge for lecturers, particularly in the absence of feedback. Understanding students’ subjective comprehension levels during online video lectures with sensor technology is yet to be thoroughly researched. This study uses eye-tracking technology to predict self-reported comprehension levels during video lectures. We recruited 20 participants from Germany and Japan who were invited to watch 50-minute lecture videos in three domains. The participants self-annotate the time segment in each lecture video where they dropout using open-source LabelStudio and answer the survey. We applied Long-Short-Term Memory (LSTM) to the preprocessed dataset and achieved an F1 Score of 0.886 for predicting binary self-annotated comprehension levels. We also introduce EyeUnderstand, the web-based application for visualizing the results of the comprehension estimation. We recruited 28 participants for the user study. As a result, 89.3% of the students and 92.9% of the lecturers confirmed that our application is practical.