Stochastic payoff-based learning in multi-agent systems modeled by means of potential games

Game theory serves as a powerful tool for distributed optimization in multiagent systems in different applications. In this paper we consider multiagent systems that can be modeled as a potential game whose potential function coincides with a global objective function to be maximized. This approach renders the agents the strategic decision makers and the corresponding optimization problem the problem of learning a maximum of the potential function in the designed game. The paper deals with a payoff-based approach to such learning. Here, we assume that at each iteration agents can only observe their own played actions and experienced payoffs. To develop the corresponding algorithms guaranteeing convergence to a local maximum of the potential function, we utilize the idea of the well-known Robbins-Monro procedure based on the theory of stochastic approximation.