Heuristics for the Development and Evaluation of Educational Robotics Systems

Contribution: This paper presents a model for educational robotics tools and a corresponding set of heuristics for their development and evaluation specifically adapted to the needs and expectations in formal education settings. Background: The increased use of educational robots in classrooms, and the steadily growing number of alternatives from which to choose, bring the requirement for appropriate methods to develop and evaluate these tools. Yet the current body of literature does not provide comprehensive frameworks that allow this question to be adequately addressed. Although previous research has studied the use of educational robotics in classrooms, there is still a lack of methods to support their development and evaluation. Intended Outcomes: An evaluation framework to support researchers, engineers, educators, and decision makers in taking informed decisions about educational robotics systems. Application Design: This paper proposes to consider activities involving educational robotics tools as a kind of “educational augmented tabletop game.” Within this framework, a set of fourteen heuristics was devised, based on literature on games and learning tools. The validity of these heuristics was examined with a heterogenous group of twelve school teachers, who tested five different educational robotics systems. Findings: The participating teachers had a high level of approval for the heuristics devised. A heuristic evaluation based on the framework proposed here appeared to more appropriately reflecting the teachers' needs than did conventional methods, namely the isolated comparison of system characteristics.