A gaze based operator instrumentation approach for the command of multiple autonomous vehicles

The control of multiple unmanned vehicles with higher levels of autonomy by a single operator has posed unique human systems integration challenges. To address this, previous researchers have attempted to assess aspects of human cognition in real time in order to expand the information exchange bandwidth to improve performance. Here, the accumulation of operator gaze data over a geospatial search area was used to refine a search area for a group of collaborative unmanned vehicles. This passively collected point of gaze input (instrumentation condition) was compared to a standard mouse input and multimodal input condition that required manual region of interest selection in a dual search area refinement task and target detection task. Data were collected in an immersive virtual environment with a head mounted display. Results indicated that gaze data were significantly correlated with predefined regions of interest, while workload from pupillometry data and target detection performance changed significantly across input conditions. These results are discussed in terms of visual attention allocation during scene recognition, and challenges with multimodal input that utilize point of gaze.