Increasing Trust in Development Processes Using Robust, Data-Driven Markov Games: An Application to PRESTIGE

In this article, we consider how to increase trust in development processes in which there is a risk for adversarial manipulation, and the adversary's objectives and resources are either ill-specified, imprecisely specified, or unknown. In such problems, we must hedge against the risk of misapprehension of attacker objectives and resources, which is further complicated in the absence of adversarial training data. We show how to model dynamic agent interaction, on the basis of partially observed or noise-corrupted data, using a partially observable Markov game (POMG) framework. We then propose a threefold heuristic solution procedure that: 1) uses the POMG to generate potential adversarial policies; 2) explicitly incorporates these adversarial policies in the construction of a robust defender policy by solving a robust dynamic program (DP); and 3) employs a probability matching heuristic in partially observable environments.