Safe Reinforcement Learning for Mixed-Autonomy Platoons: A Cooperative Control Barrier Function Approach

The control of mixed-autonomy platoons comprising both connected and automated vehicles (CAVs) and human-driven vehicles (HDVs) has attracted increasing research attention. Multi-agent reinforcement learning (MARL) appears to be a promising control strategy for its capability of managing complex scenarios in real time. However, current research on MARL-based mixed-autonomy platoon control suffers from the following limitations. First, existing MARL approaches lack theoretical safety guarantees, as they typically address safety by adding penalty terms for violations in the reward function, which does not ensure safety due to the black-box nature of RL. Second, few studies have explored the cooperative safety of multi-CAV platoons, where CAVs can be coordinated to further enhance the system-level safety involving the safety of both CAVs and HDVs. To address these limitations, we (i) design cooperative control barrier function (CBF) candidates to enhance system-level safety and (ii) integrate the safety constraints into a Multi-Agent Reinforcement Learning (MARL) framework through a differentiable quadratic programming layer. Simulation results show that our proposed control strategy can effectively enhance the system-level safety of a mixed-autonomy platoon through the cooperation of multiple CAVs.