Curriculum Design for Machine Learners in Sequential Decision Tasks

Existing work in machine learning has shown that algorithms can benefit from the use of curricula-learning first on simple examples before moving to more difficult problems. This work studies the curriculum-design problem in the context of sequential decision tasks, analyzing how different curricula affect learning in a Sokoban-like domain, and presenting the results of a user study that explores whether nonexperts generate effective curricula. Our results show that 1) the way in which evaluative feedback is given to the agent as it learns individual tasks does not affect the relative quality of different curricula, 2) nonexpert users can successfully design curricula that result in better overall performance than having the agent learn from scratch, and 3) nonexpert users can discover and follow salient principles when selecting tasks in a curriculum. We also demonstrate that our curriculum-learning algorithm can be improved by incorporating the principles people use when designing curricula. This work gives us insights into the development of new machine-learning algorithms and interfaces that can better accommodate machineor human-created curricula.