Multi-missile Cooperative Assault Strategy Model Based on Improved Proximal Policy Optimization

For the problem of multi-missile cooperative assault in distributed combat scenarios under adversarial environments, a multi-missile cooperative assault strategy model based on deep reinforcement learning is proposed. Building upon the scenario analysis of multi-missile maritime assault, a Markov decision-making model for the distribution of maritime assault firepower has been rationally hypothesized and simplified. The model employs improved proximal policy optimization algorithm for solution, with optimizations including input states normalization, output actions refinement, and the incorporation of expert knowledge into the reward function. This approach encourages missiles to approach targets, evade enemy interception, and engage undamaged vessels, thereby resolving the challenge of model convergence under sparse reward conditions. The wargaming experiment verified the effectiveness and feasibility of the model, with scores significantly surpassing those of the random fire distribution strategy and the value-based ranking fire distribution strategy. Moreover, the model has spontaneously generated command strategies that are comprehensible to commanders, including saturation attacks, targeting weak points, and coordinated assaults with various types of missiles.