Educational Sequence Mining for Dropout Prediction in MOOCs: Model Building, Evaluation, and Benchmarking

Due to the unprecedented growth in available data collected by e-learning platforms, including platforms used by massive open online course (MOOC) providers, important opportunities arise to structurally use these data for decision making and improvement of the educational offering. Student retention is a strategic task that can be supported by means of automated data-driven dropout prediction. Given the time-based nature of the collected data (user activity), these data can be viewed as sequences, and thus, sequence mining presents itself as a fitting set of techniques to automatically extract valuable insights. However, there is a lack of general guidelines for using sequence mining in specific educational settings, as well as little information on how different techniques perform in comparison to each other. We address these limitations with two main contributions. First, we propose a framework for applying sequence classification for dropout prediction in MOOCs. This framework includes two data-driven dropout definitions, the specification of data formatting and preparation tasks, and a blackprint on how to train dropout prediction models at suitable time points in the run of the course. Second, we conduct a benchmarking study of recent and well-performing sequence classification techniques, tested with different parametrizations on 47 real-life datasets from MOOCs, resulting in a comparative assessment of over 18 000 models. Our results provide insight into the performance differences between the techniques and allow us to formulate concrete recommendations toward the choice of suitable hyperparameters that have a significant influence on the predictive performance.