Cooperative Multi-Agent Advice Exchange via Topological Graph Learning

Advice exchange is a commonly used approach to enhance the performance of multi-agent reinforcement learning (MARL). It refers (requesting) agents to beneficial behaviors of (target) agents and thus facilitates efficient policy learning in a multi-agent system (MAS). However, traditional advice exchange approaches often depend on polling all agents, causing substantial communication costs and computational effort. Moreover, they adopt manually designed rules to select teacher agents, which ignore the natural topology in MAS and limit policy learning. In this paper, we propose a Cooperative Multi-Agent Advice Exchange via Topological Graph Learning (ToGAE), which entails the similarity of knowledge domains among agents in cooperative MAS. ToGAE enables agents to select their corresponding target agents with the largest knowledge domain similarity for advice exchange. The knowledge domain similarity is extracted by a two-stage graph attention network with Jensen–Shannon divergence-based training loss, and favorably reflects the functional relationship between two agents. In addition, we design a clarity-based advice acceptance scheme to avoid the blind execution of any advice, and thus further boost the efficiency of MARL. Extensive experiments show that ToGAE significantly outperforms the baseline methods in terms of the efficiency of policy learning and performance on different MARL tasks.