The Emergence of Cooperation in Multi-Agent Pursuit-Evasion Game

Pursuit and evasion conflicts are a kind of challenging problems that have important applications in many fields such as aerospace and robotics. In a pursuit-evasion game, players of the same side should cooperate in order to win the game. Existing value based methods such as VDN and QMIX cannot handle environments with continuous action space, thus cannot take kinematic constraints into consideration. Meanwhile, actor-critic methods such as MADDPG are faced with the so-called lazy agent problem, which severely damages cooperation. Moreover, the effect of all above methods is limited when reward is sparse, which makes the learning of cooperation more difficult. In this paper, a curriculum reinforcement learning with reward shaping method is proposed. By designing curriculum of different difficulty and reward function satisfying specific conditions, the proposed method helps training and boosts the emergence of cooperation among agents. Experimental results show that agents trained with the proposed method can learn more effective cooperation and show more efficiency in completing the task.