Dynamic Estimation of Mental Workload and Operator Accuracy for Time-Constrained Binary Classification Tasks

Human cognitive states, such as mental workload, play a pivotal role in decision making processes within human automation teams. Although subjective measures of mental workload can be obtained using standard questionnaires, such as the NASA-TLX, their administration is often impractical as it interferes with the primary tasks of the human operator. Therefore, it is of interest to estimate these subjective measures from less intrusive observations. Evidence suggests that mental workload is a dynamic process so incorporating historical measurements could reduce its estimation error. In addition, the estimation of operator performance in human automation teams is essential in optimizing task effectiveness and facilitating efficient resource allocation. In this work, we consider a scenario where a human and an automation solve binary classification tasks under time constraints. We present and compare different dynamic schemes to estimate the operator’s performance, i.e., classification accuracy, and its subjective ratings on subscales of the NASA-TLX questionnaire, which measure mental workload across multiple dimensions. These schemes differ in the information available for estimation. We test these schemes on data collected from a scenario, where a human and an automation perform a series of classification tasks for simulated mobile objects. Our analysis of the interaction data and the estimation schemes indicates that employing dynamic estimation for certain NASA-TLX subscale ratings leads to decreased estimation errors.