Multi-Agent Transformer Networks With Graph Attention

In addressing multi-agent reinforcement learning (MARL) challenges, Multi-Agent Transformer (MAT) has demonstrated a number of notable successes. In various benchmarks, MAT consistently showed a strong performance. A key observation in the latest MAT frameworks is MARL modeling with sequence modeling (SM) to represent the inter-agent relationships by self-attention mechanisms. This study applies graph-based modeling to represent the inter-agent relationships present in agent interactions to improve performance. To this end, we introduce the so-called MAT-GAT model, which leverages Graph Attention Networks (GAT) to allow for individualized consideration of interactions between agents. This enables MAT to pay more attention to information relative to inter-agent interactions within a cooperative MARL environment. To evaluate the performance of MAT-GAT, we conducted a series of benchmark tests across three different levels of StarCraft Multi-Agent Challenge (SMAC) tasks and the MuJoCo Half-cheetah task. The test results indicate that MAT-GAT outperforms both the original MAT and state-of-the-art baselines such as QMIX, particularly in complex environments. This demonstrates MAT-GAT’s improved performance with respect to its representation capabilities and learning.