Can an Algorithm Prepare Students for Tasks without Knowing What the Tasks Are?

We report on two consecutive randomized controlled studies that tested the implementation of a state-of-the-art neural network-based algorithm for personalizing the sequencing of content to learners based on predictive subjective difficulty level. Performance of the students who followed the algorithm recommendations were first compared to those of students who followed an expert teacher-based recommendations (study 1); then, based on the findings, we compared the impact of the algorithm recommendations to that of a baseline (non-personalized) sequence set-up by human experts (study 2). In the second study, the algorithm was successful in preparing the students to the post-test tasks equally well as the human experts were, however without knowing what these tasks were. We highlight the advantages and the limitations of the expert teacher, as well as the algorithm's ability to do no worse than the human experts.