Graph Neural Network Based Behavior Prediction to Support Multi-Agent Reinforcement Learning in Military Training Simulations

We introduce a computational behavioral model for non-player characters (NPCs) that endows them with the ability to adapt to their experiences — including interactions with human trainees. Most existing NPC behavioral models for military training simulations are either rule-based or reactive with minimal built-in intelligence. Such models are unable to adapt to the characters' experiences, be they with other NPCs, the environment, or human trainees. Multi-agent Reinforcement Learning (MARL) presents opportunities to train adaptive models for both friendly and opposing forces to improve the quality of NPCs. Still, military environments present significant challenges since they can be stochastic, partially observable, and non-stationary. We discuss our MARL framework to devise NPCs exhibiting dynamic, authentic behavior and introduce a novel Graph Neural Network based behavior prediction model to strengthen their cooperation. We demonstrate the efficacy of our behavior prediction model in a proof-of-concept multi-agent military scenario.