Evaluation of learner's skills in the context of dynamic and complex systems

The air traffic control (ATC) operational skills are developed using simulation of authentic problems. Such authentic problems have enormous potential for learning as they are real and will probably arise in the controller's real work. Resolving authentic problems on a simulator encourages transfer by demonstrating when the knowledge is useful and by showing the effect of the learner's actions. However this process is difficult, since authentic problems tend to require a wide range of knowledge which often lead the learner in activities beyond their abilities. Some support is required for learning from simulation to occur. Sometimes this support is provided by human tutors (in ATC training for example this support is provided by a human instructor sitting at the back of the controller during the running of an exercise on the simulator), this support could also be given by a computer learning environment. This paper presents an intelligent simulator-based learning environment for air traffic controllers which provides a protected and guided environment to the learner. One central element of this environment is the ability to evaluate the learner's skills. We describe our approach to the evaluation of learner's skills, which uses task performance as well as situation complexity and degree of task practice as a means for building an overlay model of the learner.