Game-Theoretic Optimization of Active Orbital Defense via Multi-Agent Reinforcement Learning

This paper investigates an active defense-oriented orbital pursuit-evasion-defense game problem involving three spacecraft. A reinforcement learning framework is proposed to train active defense policy under multiple constraints including manoeuvrability limitations and fuel limitations. Unlike existing approaches, the algorithm in this paper integrates a novel predict-reward-detect mechanism that embeds long-term trajectory prediction into the multi-agent proximal policy optimization algorithm training process. This mechanism mainly solves the reward sparsity and state sparsity problems caused by long decision cycles. Active defense policy is trained through the self-playing method. Simulation experiments demonstrate that the proposed method can achieve the 96% defense success rate. For different initial conditions, the proposed method maintains efficient defense performance, effectively verifying the effectiveness of active defense policy in complex confrontation scenarios.