Deep Learning Method for Estimating Perplexion in Learners by Educational Support Robots

Educational support robots have been studied in recent years. This paper focuses on teacher-type robots that behave like teachers. Prior research has demonstrated that teacher-type robots that provide educational support based on the learning achievements of the learners also provide a high learning effect in problem-solving situations. However, conventional teacher-type robots provide learners with learning support when a problem is presented or when the learner requests learning support by pressing a button. It is therefore not possible for conventional teacher-type robots to estimate when a learner is troubled (perplexed) or to autonomously provide learning support when the learner cannot solve a problem. A teacher-type robot that autonomously provides support can allow more effective learning, thereby improving the learning effect. In this study, we developed a teacher-type robot that estimates when a learner is perplexed and autonomously provides learning support. In this paper, we propose the use of a neural network to estimate when a learner is perplexed based on the learner's facial expressions when solving a problem. The proposed method uses a neural network to estimate the level of perplexion based on the probability distribution of seven basic emotions estimated from facial images of learners when applying a CNN model.