An Intelligent Car-Following Model Based on Multi-Step Deep Reinforcement Learning

With the development of vehicular networking technology, the development of car-following model that serves for intelligent and/or autonomous driving has been an interested topic. Deep reinforcement learning (DRL) based models are attracting considerable attentions. However, due to the complex traffic environment, development of an efficient car-following model is a challenge issue in terms of extended training times, low sample utilization rates, and poor stability. To deal with these issues, this work proposes a reward processing mechanism that considers the cumulative impact of multi-step reward. The proposed DRL algorithm shows expedited learning speed, enhanced sample utilization and effectively resolved the challenges encountered during training. To evaluate the proposed car-following model, SUMO, an open source for micro, multi-modal traffic simulation is used to construct a realistic driving environment. Finally, the results with respect to several driving scenario validate the model, and the comparation results to the widely discussed intelligent driver model (IDM) model show the advancements of the proposed model.