Explainability for Unmanned Aerial Vehicle Control via Multi-Objective Reinforcement Learning

Reinforcement Learning (RL) is a widely used machine learning technique allowing Unmanned Aerial Vehicles (UAVs) to learn, via interaction, behavior that maximizes a scalar-valued reward function. Many real-world and simulated warfare scenarios require UAV s to perform complex behavior while considering multi-ple competing objectives such as fuel conservation and agility. However, standard RL techniques lack de-fined analytic methods for determining how modifications to the reward function may result in sub-optimal agent behavior for these competing objectives. Multi-Objective Reinforcement Learning based on Decomposition (MORL/D) splits multi-objective problems into several single-objective sub-problems, allowing for parallel optimization of conflicting objectives and production of a set of optimal behaviors. In this study, we find agents trained to automate UAV behavior using a MORLID-based approach exhibit both high-quality diverse strategies, as evidenced by the expected utility and hypervol-ume of their associated Pareto Sets, and comparable or superior performance to heuristic agents in both offensive and defensive battle management scenarios with multiple conflicting objectives. Additionally, this study provides intuitive visualization methods that clarify the contribution of individual rewards for MORL agents. These findings indicate MORL/D is a promising framework for RL-based UAV Command and Control in complex battle management scenarios where balancing conflicting goals is crucial for mission success.