Decentralized reinforcement social learning based on cooperative policy exploration in multi-agent systems

Coordination problems including miscoordination and relative overgeneralization are difficult to overcome especially in dynamic and stochastic environments. In the practical scenario, there may be a large number of agents, and the interactions between agents may be sparse and unfixed. In this paper, we study the coordination problems and stochastic rewards under the social learning framework where there are a group of agents and each agent behaves independently and interacts with another agent randomly chosen from the group. We are looking for a learning technique that makes all agents learn a consistent optimal policy in a two agents game with pathology of coordination problems or stochastic rewards under such a framework. A new algorithm named Decentralized concurrent learning and cooperative policy exploration (DCL-CPE) is contributed, which possesses the ability to overcome the coordination problems and the stochastic rewards via local interaction under the social learning framework. Empirical results for several cooperative games are presented to show the superiority of our algorithm.