Local Coordination in Multi-Agent Reinforcement Learning

This paper studies cooperative multi-agent reinforcement learning problems where agents pursue a common goal through their cooperation. Because each agent needs to act individually on the basis on its local observation, the difficulty of learning depends on to what extent information can be exchanged among agents. We extend value-decomposition networks (VDN), a framework requiring the least communication, by allowing information exchange within a local group and present residual group VDN (RGV). We empirically show that the performance of RGV is better than VDN and other state-of-the-art methods in the predator-prey game. Also, on three tasks in the StarCraft Multi-Agent Challenge, RGV showed comparable performance with more sophisticated methods utilizing more information or communication. Therefore, our RGV is an alternative method worth further research.