Collaborative Control of Vehicle Platoon based on Deep Reinforcement Learning

We propose a novel vehicle platoon collaborative control algorithm based on deep reinforcement learning. Aiming at the slow convergence speed of traditional reinforcement learning (RL) algorithms, a deep deterministic policy gradient (DDPG) algorithm based on guidance (named GuidDDPG) is suggested. The adaptive cruise control (ACC) model is introduced as the guidance of the early training stage of the RL agent and the action synthesizer is designed to integrate the ACC model and the RL agent action output to overcome the problem of slow algorithm convergence. The goal oriented reward function is constructed to train agents to make decisions, which combines the advantages of RL algorithm and ACC model. The simulation results show that the proposed GuidDDPG algorithm has faster convergence speed and higher efficiency than the DDPG algorithm and twin delayed DDPG (TD3) algorithm. For the mixed vehicle platoon, the unified vehicle platoon control model is developed. The GuidDDPG algorithm is used to build a generalized RL agent to learn the optimal control strategy. The influence of different penetration rates and different distributions of human-driven vehicles on the mixed vehicle platoon is analyzed through simulation.