Learn to Coordinate with Generic Non-Stationary Opponents

Learning to coordinate with non-stationary opponents is a major challenge for adaptive agents. Most previous research investigated only restricted classes of such dynamic opponents. The main contribution of this paper is twofold: (i) A class of generic non-stationary opponents is introduced. The opponents keep mixed strategies which change with less regularity. Its showed that the independent reinforcement learners (ILs), which have neither prior knowledge nor opponent models, cannot coordinate well with this type of opponent, (ii) A new exploration strategy, the DAE (detect and explore) mechanism, is tailored for the ILs in such coordination tasks. This mechanism allows the ILs dynamically detect changes in the opponents behavior and adjust their learning rate and exploration temperature. It's showed that ILs using this strategy are still able to converge in self-play and are able to coordinate well with the non-stationary opponents