Single Pilot Operations with AR-Glasses using Microsoft HoloLens

This paper presents a conceptual design of an additional assistant system with AR-Glasses for single-pilot operations of commercial transport aircraft. In particular, an analysis regarding the use of the Microsoft HoloLens as a guidance device and co-pilot compensation, to perform useful and timely alerts based on airline emergency procedures. The augmented reality (AR) device was introduced to raise the ability to handle complex work flows under time pressure by adding artificial visual guidance to the crew member. The goal was to enhance situation awareness and confidence taking carefully time critical decisions without the support of a second crew member. The test scenario included an engine fire in a single cockpit operation environment during cruise flight. For the survey, 24 experienced and type rated active Airbus A320 pilots have been compared in two groups to each other using an Airbus A320 fix based simulator. Within the test scenario, the reaction time initiating the standardized action items (based on Airbus A320 quick reference handbook) have been recorded. The first group had only support by the airplane-side systems (e.g. ECAM display), while the second group used the AR-glasses. Compared to the conventional airplane-side systems support to deal with engine fire situations, direct visual instructions have been given by the AR-glasses and have been always linked to a displayed three-dimensional visual marker. This should immediately inform the pilot where the focus of attention must be on and thus ensure that the actions are carried out more quickly. By guiding the pilot through visual instructions, manual scanning of the cockpit parameters have been unnecessary. The direct instructions in the glasses and the additional information expected to shorten the reaction time. At the same time this should relieved the pilot and reduce the workload in order to get a more efficient human machine system and enhanced situation awareness.