Modeling and Influencing Human Attentiveness in Autonomy-to-Human Perception Hand-offs

It is not uncommon for autonomous systems (e.g., self-driving cars) to require the timely intervention of a human operator to ensure safe operation. It is important to design these systems such that the human is brought into the decision-making loop in a manner that enables them to make a timely and correct decision. In this paper, we consider one such ap-plication, which we refer to as the perception hand-off problem, which brings the driver into the loop when the perception module of an Autonomous Vehicle (AV) is uncertain about the environment. We formalize the perception hand-off problem by designing a Partially Observable Markov Decision Process (POMDP) model. This model captures the latent cognitive state (attention) of the driver which can be influenced through a proposed query-based active information gathering (AIG) system for Human-Machine Interface (HMI). We design a web-based human study to identify the model parameters, and demonstrate the impact of the proposed HMI system. Results from this study show that the state of attentiveness does indeed impact the human performance, and our proposed active information gathering (AIG) actions, i.e., queries to the human driver, result in 7% faster responses from the human. Simulations with the identified POMDP model show that a learnt policy for deploying the AIG actions improves the percentage of correct responses from the human in the perception hand-off by around 5.4%, outperforming other baselines while also using fewer of these actions.