Performance-based Data-driven Assessment of Trust

The likelihood of an agent’s success in achieving its goals and subgoals under specific operational constraints is an important factor in performance-based trust. Trust management systems are being proposed to monitor performance and other trust-related system variables to improve operational efficiency. We present in this paper an exploratory study on the feasibility of assessing the performance of drone operators in a target-finding task through subjective and objective evaluations of the drone’s flight data. Thirty-nine participants were trained in using a teleoperated drone to find specific alphanumeric targets arranged on the laboratory floor and thereafter performed the task three times in separate three-minute runs. Following this, they made subjective assessments of other operators’ performance based on viewing the flight paths flown. Linear mixed effects modelling of subjective performance assessments showed either non-significant or weak relationships between subjective assessments and actual operator performance. Binomial logistic regression modelling showed significant relationships between features of the drone’s flight pattern and operator performance. These results could inform the development of data-driven, performance-based trust management systems for use in collaborative human and machine environments.