A Real-Time Analysis of Human Performance in Interactive and Adaptive Mixed-Reality Simulation

Intelligent adaptive systems generally use inferential or computational methods to identify and adapt to the user’s internal state of interest. Toward an adaptive human-in-the-loop simulation, his article defines, implements, and validates an analytic scheme to estimate real-time, task-specific, and comprehensive human performance encompassing perception and action. The proposed scheme builds on the previous work of mixed reality (MR) simulation app designed to perform a virtual object-following task for screening and diagnosis of neurocognitive impairment. The proposed human performance model adopts the Fitts’ paradigm of speed-accuracy tradeoff in the context of continuous, rapid aimed movements. For validation, the performance quantification of eleven healthy subjects over two repeated sessions was statistically analyzed. The results potentially confirm a qualification of the metric in terms of stability (by not showing statistically significant variance over repetition), but for an accurate estimate of individual performance, the scheme needs to be adjusted with its segmentation threshold.