Model-based deep reinforcement learning for CACC in mixed-autonomy vehicle platoon

This paper proposes a model-based deep reinforcement learning (DRL) algorithm for cooperative adaptive cruise control (CACC) of connected vehicles. Differing from most existing CACC works, we consider a platoon consisting of both human-driven and autonomous vehicles. The humandriven vehicles are heterogeneous and connected via vehicleto-vehicle (V2V) communication and the autonomous vehicles are controlled by a cloud-based centralized DRL controller via vehicle-to-cloud (V2C) communication. To overcome the safety and robustness issues of RL, the algorithm informs lowerlevel controllers of desired headway signals instead of directly controlling vehicle accelerations. The lower-level behavior is modeled according to the optimal velocity model (OVM), which determines vehicle acceleration according to a headway input. Numerical experiments show that the model-based DRL algorithm outperforms its model-free version in both safety and stability of CACC. Furthermore, we study the impact of different penetration ratios of autonomous vehicles on the safety, stability, and optimality of the CACC policy.