Safe Reinforcement Learning Using Model Predictive Control with Probabilistic Control Barrier Function

Practical applications of reinforcement learning (RL) often demand that the agents explore safety by satisfying designed constraints. The constraints for safety cannot be satisfied with probability 1 when an unbounded stochastic uncertainty is present. One way is to relax the hard constraints into chance constraints and to consider safe RL with chance constraints. This paper addresses safe RL with chance constraints by Model Predictive Control (MPC) with Probabilistic Control Barrier Function (PCBF). MPC with PCBF is used as a function approximator to deliver the policy that satisfies the demanded chance constraints. RL is used to optimize the parameters in MPC with PCBF to improve the closed performance. We prove that using PCBF as a constraint in MPC ensures the safety imposed by the chance constraints. A scenario-based algorithm is designed for the proposed safe RL implementation. A quadrotor system control problem in turbulent conditions has been used as a numerical example to validate the proposed safe RL method.