A Multi-Modular Sensor Fusion and Decision-Making Approach for Human-Machine Teaming

Human-Machine Teaming (HMT) will require real-time evaluation of human and machine states and the integrated optimization of their collective performances during their mission task. This paper presents the development of a multi-modular sensor fusion and decision-making (MMSF-DM) architecture that uses human multi-task performance data and human physiological data in the simulated operation of a machine to determine (a) the subtask difficulty and (b) the overall task difficulty for the human. The simulated machine was a Tennessee State University developed version of the Multi-Attribute Task Battery (TSU-MATB) simulator previously developed by NASA. The MMSF-DM is based on the Conscious Architecture for State Exploitation (CASE). The MMSF-DM incorporating artificial neural networks (ANN) and adaptive neuro-fuzzy inference system (ANFIS) methodologies was shown to correctly characterize the overall task difficulty level corresponding to the difficulty level in the human subject experimental data.