Projection-based Policy Iteration using Control Barrier Function

The control barrier function (CBF) method offer a flexible paradigm for encoding closed-loop system safety by constructing a half-plane state-dependent input constraint. In this paper, we introduce a class of projection-based safe policy iteration algorithms designed to derive control policies that encode both system safety and stability concurrently. First, a Hamilton-Jacobi-Bellman (HJB) for affine nonlinear dynamics with CBF constraints is established. We subsequently derive optimal state feedback policies using the Karush-Kuhn-Tucker (KKT) conditions. Then, initializing from an admissible control policy, the recursive policies obtained from each policy evaluation step are projected into a safe policy set, which eventually converges to a safe and optimal policy. The convergence of the algorithm is analyzed in detail. Finally, the efficacy of the proposed algorithm is demonstrated through numerical simulations.