What Do You Want From Me? Adapting Systems to the Uncertainty of Human Preferences

Autonomous systems, like drones and self-driving cars, are becoming part of our daily lives. Multiple people interact with them, each with their own expectations regarding system behaviour. To adapt system behaviour to human preferences, we propose and explore a game-theoretic approach. In our architecture, autonomous systems use sensor data to build game-theoretic models of their interaction with humans. In these models, we represent human preferences with types and a probability distribution over them. Game-theoretic analysis then outputs a strategy, that determines how the system should act to maximise utility, given its beliefs over human types. We showcase our approach in a search-and-rescue (SAR) scenario, with a robot in charge of locating victims. According to social psychology, depending on their identity some people are keen to help others, while some prioritise their personal safety. These social identities define what a person favours, so we can map them directly to game-theoretic types. We show that our approach enables a SAR robot to take advantage of human collaboration, outperforming non-adaptive configurations in average number of successful evacuations. CCS CONCEPTS • Computer systems organization $\rightarrow$Robotics; • Human- centered computing $\rightarrow$Collaborative interaction. ACM Reference Format: Carlos Gavidia-Calderon, Amel Bennaceur, Anastasia Kordoni, Mark Levine, and Bashar Nuseibeh. 2022. What Do You Want From Me? Adapting Systems to the Uncertainty of Human Preferences. In New Ideas and Emerging Results (ICSE-NIER’22), May 21-29, 2022, Pittsburgh, PA, USA. ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/3510455.3512791