Stochastic zero-sum nash games for uncertain nonlinear Markovian jump systems

In this paper, a novel adaptive learning technique is proposed to solve a stochastic zero-sum Nash game with partially unknown nonlinear systems for which the lengths of time intervals that the system spends in each mode are independent random variables with exponential distributions, i.e. the environment and the cost matrices depend on the outcome of a Markov chain. We first formulate the problem by using an optimal stopping process and then provide a verification theorem for stopping zero-sum games. A structure of 2 actors and 1 critic approximators are used to approximate the saddle-point policies and the optimal cost respectively. Effective tuning laws are proposed to solve the stochastic Nash game problem while also guaranteeing closed-loop stability with the use of rigorous Lyapunov-based stability proofs. Finally, a numerical example is used to illustrate the effectiveness of the proposed approach.