Learning Distributed Coordinated Policy in Catching Game with Multi-Agent Reinforcement Learning

Although learning-based methods such as reinforcement learning have been applied to multi-agent systems design successfully, it is still difficult to learn efficient coordinated policies for agents in partially observed environment settings. Centralized learners contain much more information, but add more complexity, while independent learners suffer from partial observation. To address these problems, we propose a directed multi-agent actor-critic algorithm to directly learn the coordinated policy from experience. The directed critic model can obtain all information including global information and actions, which provides effective learning signals for distributed learning actors. We take Multi-Agent Catching Game as the test scenario, where the task is to coordinate multiple moving paddles to catch balls dropping from the top of the screen. We perform several experimental evaluations and show that our method leads to superior results in learning performance, coordination effect and scalability, compared with both centralized and independent learning approach.