Attention Guidance for Tower ATC Using Augmented Reality Devices

In 2021 Royal NLR carried out innovative technology experiments on their high-fidelity real-time air traffic control simulation and validation platform, NARSIM. These experiments were part of the SESAR 2020 project Digital Technologies for Tower (DTT). The technology option that was investigated focused on advanced HMI interaction modes for aerodrome tower controllers. More particular, Attention Capturing and Guidance strategies with an Augmented Reality device, the Microsoft HoloLens 2™, were evaluated inside an aerodrome control tower environment for Amsterdam Airport Schiphol, one of the major European hub airports.The NARSIM environment consisted of a realistic but downscaled presentation of the airport with two tower controller working positions emulating current tower systems. Such a set-up allowed researchers to focus their work on the application of Augmented Reality with the introduction of (virtual) aircraft labels as well as special symbology and auditory cues for capturing and guiding tower controller attention in the case of critical events. Several typical attention-critical events that may occur at an airport, such as go-around operations and runway incursions, were orchestrated by a team of NLR experts and presented to the tower controllers while they were operating traffic as usual. Human performance and ATC operational experts observed and analyzed the simulations.This paper describes the steps taken and the challenges encountered when integrating the HoloLens inside the NARSIM Tower environment. Furthermore, it explores the proposed operational concept for Attention Capturing and Guidance with the HoloLens and how it was realized inside the device. The results of the technical evaluation activity with two experienced air traffic controllers are described in detail. These results came to the conclusion that the device in combination with the concept was a favorable addition to the controller working environment. While desired technical performance improvements, mostly related to user comfort and general adjustments, depend on further vendor development, the used HoloLens was seen as a technically useful device for implementing prototype applications for Attention Capturing and Guidance with aural and visual cues.In the final sections of the paper, an outlook into the expected future use of Augmented Reality devices in conventional control tower environments is given.