Distributed Tracking of Correlated Equilibria in Regime Switching Noncooperative Games

We consider a class of regime-switching noncooperative repeated games where agents exchange information over a graph. The parameters of the game (number of agents, payoffs, information exchange graph) evolve randomly over time according to a Markov chain. We present a regret-based stochastic approximation algorithm with constant step-size that prescribes how individual agents update their randomized strategies over time. We show that, if the Markov chain jump changes on the same timescale as the adaptation rate of the stochastic approximation algorithm and agents independently follow this algorithm, their collective behavior is agile in tracking the time-varying convex polytope of correlated equilibria. The analysis is carried out using weak convergence methods and Lyapunov stability of switched Markovian differential inclusions.