A multi-agent coordination framework based on Markov games

Based on the analysis of the reinforcement learning and Markov games, the paper proposes a layered multi-agent coordination framework. Based on the multi-agent's interaction of competition and cooperation, this coordination framework adopts the zero-sum Markov game in higher layer to compete with opponent and adopts the team Markov game in lower layer to accomplish the team's cooperation. This coordination framework is applied to Robot Soccer. The results of the experiment illuminate that our proposed method is better than the traditional multi-agent learning.