Multi- Touch Touch Screens on the Flight Deck: The Impact of Display Location, Display Inclination Angle and Gesture Type on Pilot Performance

Under NextGen operations, multifunction flight deck controls such as touch screens are being used in new ways. Since many of the multifunction touch screen controls (TSCs) being considered for the flight deck are multi-touch, it is important that their usability and human factors considerations be studied in realistic aviation environments. A study was conducted to help understand the impact of multi-touch display location, display inclination angle, and gesture type on task performance, as well as workload and fatigue, for single pilots flying a flight simulator in moderate turbulence conditions. Two preliminary studies were conducted in order to inform a larger single pilot multi-touch study. A preliminary gesture study explored seventeen touch gestures and identified a subset of nine gestures to use in the larger single pilot study. A preliminary inclination study assessed the impact of inclination angle on pilot performance while using touch gestures. The main single-pilot study sought to further explore pilot performance using multi-touch gestures by incorporating the results from both preliminary studies in the context of flight simulation with turbulence. Electromyography (EMG) was used to capture muscle activations at the forearm and shoulder as an indicator of potential fatigue. Tasks included gestures performed on instrument approach procedures (IAP), maps, on a menu and application driven interface, and on a fuel balance interface. Dependent measures included time on task; number of touches to completion, perceptions of workload and fatigue, readability rating, objective fatigue via EMG, maintenance of flight indices, and ratings questionnaires. The results of this study revealed various human factors implications associated with the application of multi-touch screen controls in the flight deck. Generally, both time on task and the number of gesture attempts before successful task completion were elevated for gestures requiring more than two digits. In addition, gestures requiring more than two digits have greater potential to obscure touch targets and flight deck display information. These gestures were often awkward to complete even in a static environment; this may worsen in flight, especially in turbulent conditions. For completing touch screen tasks with gestures, pilots preferred four of the presented display inclination angles. Touch gestures performed at the upper multifunction display (MFD) location often required pilots to “lean forward” or stretch by bending at the waist and lifting their back off the seat to achieve the reach needed. This implies greater propensity for issues with fatigue. The results are intended to help inform the development of guidelines and recommendations for the integration of multi-touch screen controls into flight decks.