Modeling Safe and Smooth Car-Following Behavior: An LSTM-Attention Based TD3 Approach

Deep reinforcement learning (DRL) is an effective and popular way to study car-following behavior due to its ability to deal with complex and dynamic environment. However, the performance of DRL based car-following models is degraded by inadequate network structure and unreasonable reward function design. In order to solve this issue, this study creatively proposes a car-following model which incorporates the long short-term memory (LSTM) network and the attention mechanism into the Twin Delayed Deep Deterministic Policy Gradients (TD3) architecture. In our model, the historical driving information is considered to simulate the memory effect of human drivers. Additionally, to reduce the risk of rear-end collisions, a new safety-oriented driving feature is designed and integrated into the reward function. The experimental results reveal that our model can effectively speed up model convergence, while also obtaining the optimal results with the consideration of driving safety, efficiency, and comfort.