Initially Excited Learning of Discrete-Time Multiplayer Games

Data-storage-based reinforcement learning (RL) algorithms for dynamic games typically impose a persistent excitation (PE) condition to guarantee convergence. However, this requirement can be overly restrictive in practical implementations. To address this limitation, this paper proposes a novel online RL algorithm for approximating the Nash equilibrium solutions in discrete-time multiplayer nonzero-sum games under a relaxed initial excitation condition, thereby avoiding the need for the PE assumption. Different from existing methods that rely on data storage mechanisms and full-rank conditions, the proposed algorithm incorporates an online verification scheme that evaluates the positivity properties of an augmented matrix. This mechanism ensures initial excitation without requiring heuristic data collection designed to satisfy the PE condition. Furthermore, we provide a rigorous analysis to establish the convergence of the designed algorithm. Finally, comparative studies validate the effectiveness of the proposed approach.