Discrete-time Stochastic Multi-agent Graphical Games with Multiplicative Noise: Reinforcement Learning Solutions

This paper considers a discrete time stochastic multi-agent graphical games problem with multiplicative noise based on reinforcement learning. The Bellman optimality equations for multi-agent graphical games are obtained by using the optimality principle. Through the stability analysis, it is proved that the solutions of the equations converge to Nash equilibrium. Since the coupled Bellman equation is difficult to solve, the value iteration heuristic dynamic programming (HDP) algorithm is designed. To solve multi-agent graphical games online, the HDP algorithm based on the actor-critic framework is designed to find the approximate solutions, and the effectiveness of the algorithm is verified by a simulation example.