Resolution Requirements for Virtual and Mixed Reality Pilot Training

Virtual and mixed reality technologies have improved significantly in recent years. Their benefits outweigh their limitations, and they are being widely adopted by all advanced air forces for pilot training around the world. One of the main aspects discussed with their use is the human eye acuity and the impact of fixed focal length optics introduced by Head Mounted Displays (HMDs). This paper describes the relationship between average pilot acuity and the most commonly used HMDs, comparing their capabilities with training requirements. It emphasizes the importance of matching technological capabilities with visual acuity requirements for specific training, focusing on concepts such as Minimum Recognizable Acuity (MRA) and Pixel Per Degree (PPD). Minimum Recognizable Acuity (MRA) measures a pilot's ability to see and distinguish small details. In today's modern fighter aircraft, technology plays an important role in enhancing the pilot's situational awareness. Head-Up Displays (HUDs) and other electronic systems provide critical information and data, making it imperative to correlate MRA requirements with the concept of Pixel Per Degree (PPD), which quantifies the visual resolution of display systems. Pixel Per Degree (PPD) is a measure of how many pixels are present in one degree of a pilot's field of view. It is critical in the context of MRA because it determines the level of detail a pilot can see on a display. Achieving the optimal balance between the two is challenging, as increasing PPD on displays can result in smaller, more detailed information, but this must be balanced with the pilot's ability to recognize and process this information effectively. The paper also highlights the complex relationship between MRA and PPD in creating effective VR/MR training systems. It points out that pilots’ visual acuity is tested for mission success and safety, and that the design of VR headsets must meet these requirements. The study summarized in the paper compares different VR/MR headsets used by the U.S. Air Force and U.S. Navy for immersive training and explains technical factors that influence visual acuity, such as real pixel distribution, rendering pipeline, warping algorithm, and other aspects with direct visual impact. The interplay between Minimum Recognizable Acuity (MRA) and Pixel Per Degree (PPD) values is essential to designing cockpit displays that meet the stringent visual requirements of military pilots. All of this also applies to simulation and training systems, as they must accurately emulate the real cockpit for all subsequent heads-up displays, multi-function displays, and real-world views with simulated terrain, objects, and entities. The paper further describes the role of cameras embedded in HMDs and their specification requirements for immersive mixed reality training with physical cockpit simulator. It classifies different training scenarios based on fidelity requirements and outlines a process for selecting suitable VR/MR devices for specific training, emphasizing the importance of performing side-by-side comparisons. In conclusion, the paper acknowledges the advances in VR/MR headset technology and recognizes the existence of training scenarios that are fully suitable for immersive training, but notes the need for higher resolution displays, advanced optics, and better performing cameras to fully simulate the realistic visual environments required for a variety of pilot training scenarios. The study aims to assist in defining the technical requirements for VR/MR pilot training, taking into account the main factors influencing visual fidelity. (Abstract)