Exploration of human factors issues within a future separation assurance concept

The human factors (HF) impact of sweeping changes in the roles of human operators, as well as the introduction of new technologies, are being studied in NASA's Next Generation Air Transportation System (NextGen). As part of a NASA funded project [1], a walkthrough technique was constructed to examine the effectiveness of using a low-cost method for looking at NextGen concepts in detail within a concrete operational context. A ground-based Separation Assurance (SA) concept was chosen as a specific example and its instantiation for a high-level en route air traffic controller position was selected as the focus. Test run recordings from a previous study [2] provided four off-nominal events as stimuli for our walkthrough. Each event was analyzed to identify its progression due to an action by the controller or the automation. Based on reviews of four NextGen research concepts [1, 3], 18 HF themes were selected as key areas affected by introducing NextGen automation for the SA function. These encompassed cognitive and organizational topics including attention, workload and job responsibilities. A walkthrough was constructed by applying these themes as questions to relevant points in a set of events. Six retired controllers watched each event three times. First, the event was played in real time; second, the event was stepped through and a question with a cognitive theme was asked; and third, the event was stepped through again and an organization question was asked. Participants' answers were recorded and later transcribed. Qualitative analyses selected questions that addressed the 18 themes. Results indicated valuable unique operational insights into the problems for the NextGen SA concept not previously available through human-in-the-loop simulations alone. Given that other concept exploration methods are resource-intensive (e.g. human in the loop), the cognitive walkthrough was found to be a low cost and reasonably rapid method for exploring HF issues. The use of a dynamic ldquostoryboardrdquo to provide the stimulus for the walkthrough questions, while moving away from the original cognitive walkthrough method, was considered to be essential in the domain due to the spatial and dynamic nature of controllers' expertise.