Assessing Pilot Workload during Takeoff and Climb under Different Weather Conditions: A fNIRS-based Modelling using Deep Learning Algorithms

Excessive workload caused by weather conditions may increase pilot errors and flight risk. The assessment of pilot workload using pilots' physiological data is a potential method to address this issue. We conducted a flight simulator study involving 18 cadet pilots with real flight experiences ranging from 232 to 250 hours and used non-invasive functional Near-Infrared Spectroscopy (fNIRS) to measure the pilots' brain activity. Pilots' subjective National Aeronautics and Space Administration-task load index (NASA-TLX) ratings of

 0000-0000 © 2022 IEEE

workload were also recorded. The tested flight maneuvers included a total of 54 takeoff and climb tasks under different weather conditions. Over 1100 features covering three hemoglobin signals and four brain cortexes were extracted from the fNIRS data. A statistical analysis was carried out on the subjective ratings and fNIRS features, and a weighting analysis was applied to the selected fNIRS features that were highly sensitive to pilot workload levels. A Convolutional Neural Network (CNN) with an attention mechanism (CNN-Attention) was built as a classifier for assessing pilot workload and compared with CNN, Deep Neural Network (DNN), eXtreme Gradient Boosting (XGBoost), and Random Forest (RF) models. The results suggested that pilot workload was highly associated with three hemoglobin measures as well as the activities of different brain regions, including the prefrontal cortex, motor cortex, and occipital cortex, and the CNN-Attention performed best. The findings from this study could provide a reference for optimizing pilot training systems and improving flight safety under different weather conditions