Recurrent MADDPG for Object Detection and Assignment in Combat Tasks

With the development of artificial intelligence, multiagent algorithms have been applied to many real-time strategy games. Making plans on the human being is gradually passing away, especially in combat scenarios. Cognitive electronic warfare (CEW) is a complex and challenging work due to the sensitivity of the data sources. There are few studies on CEW. In the past, wargame simulations depended on differential equations and war theory, which resulted in high time and human resource costs. In the future, as other artificial intelligence theories are developed, artificial intelligence will play a more critical role in wargames. The capabilities of multiagent modeling to describe complex systems and predict actions in dynamic environments are superior to those of traditional methods. In this paper, we use a 3D wargame engine from China Aerospace System Simulation Technology Co., Ltd. (Beijing) named All Domain Simulation (ACS), which supports land, sea, and air combat scenarios, to simulate combat. In the simulations, there are several unmanned air vehicles (UAVs) as attackers and several radar stations as defenders, and both have the ability to detect the others. In the game, several UAVs need to learn to detect targets and track targets separately, and we train the UAV's behavior by well-designed reward shaping and multiagent reinforcement learning (MARL) with LSTM. We improved the RDPG algorithm and merged the MADDPG and RDPG algorithms. From the experimental results, we can see that the effectiveness and accuracy of the algorithm have been greatly improved.