Homotopy-Based Single-Loop Policy Iteration for Zero-Sum Games of Unknown Linear Systems

The simultaneous policy update algorithm (SPUA) has been extensively studied for linear zero-sum games due to its efficient single-loop iteration. However, selecting an appropriate initial matrix for the SPUA to satisfy the Newton-Kantorovich conditions and ensure convergence remains a challenging task, especially in model-free settings. In this letter, a homotopy-based single-loop policy iteration method is proposed for linear zero-sum games. The designed method is guaranteed to converge using only initial stabilizing controllers. And a homotopy-based approach is employed to compute these initial stabilizing controllers through a series of policy improvement iterations. This method enables efficient single-loop iterations without requiring an initial matrix guess, a predetermined stabilizing controller, or knowledge of system dynamics. Simulations are conducted, and the numerical results also demonstrate the effectiveness of the proposed method.