Maintaining cooperation in homogeneous multi-agent system

During multi-agent interactions, robust strategies are needed to help the agents to coordinate their actions to achieve efficient outcomes. A large body of previous work focuses on designing strategies towards the goal of Nash equilibrium, which can be extremely inefficient in many situations such as prisoner's dilemma game. A number of improved algorithms based on Q-learning have been developed recently for agents to achieve mutual cooperation in games like prisoner's dilemma. However, almost all of them involve only two agents playing the same game repeatedly. However, this may not reflect the real scenario in practical multi-agent interaction situations. In practical multi-agent environments, each agent may interact with multiple agents at the same time and each agent may only be allowed to interact with a specific set of agents, which is determined by the system's underlying topology. In this paper, we propose a learning framework by taking into consideration the underlying interaction topology of the agents. We show that the system can maintain certain level of cooperation though the agents are individually rational. To better understand this phenomenon, we also develop a mathematical model to analyze the dynamics resulting from the learning framework. The theoretical results of the mathematical model are shown to be able to successfully predict the transition point and the expected behaviors of the system compared with the simulation results.