A method for evaluating project management competency acquired from role-play training

The information technology industry in Japan has required universities to provide project management education. In Tokyo University of Technology, role-play training has been carried out as part of project management education. The role-play scenarios necessary to run role-play exercises have been created in accordance with the ADDIE (Analysis, Design, Development, Implementation, Evaluation) model. This paper describes a method for evaluating project management competency that learners gain through role-play training conducted using the scenarios. Competency in project management is assessed from a learner's behavior characteristics in taking an appropriate action when needed. We first examined the quality of the role-play scenarios by using a design checklist based on Goal-Based Scenarios (GBS). In addition, we analyzed the behavior of each learner during a role-play exercise by using rubrics based on how the user behaved. A high correlation was found between the acquired skill with which learners generally played the role assigned to them in role-play training and the level of quality of the role-play scenario. Based on the analysis results, we will propose a method for helping learners to be able to take effective action by providing appropriate advice from a software agent and feedback from a teacher, along with use of the GBS checklist.