Personalized Car-Following Strategy Based on Convolutional Variational Autoencoder and Mildly Conservative Q-Learning

With the development of intelligent vehicles, the importance of imitating, learning, and optimizing human driving strategies is increasingly recognized. Offline reinforcement learning offers a promising approach, capable of directly extracting driving strategies from diverse real-world datasets, particularly in critical vehicle safety systems, where it provides advantages in interpretability and transferability. We propose a personalized car-following strategy framework based on a one-dimensional convolutional variational autoencoder (conVAE) and mildly conservative Q-learning (MCQ). The conVAE extracts subtle feature differences among drivers, classifying their driving styles as the foundation for strategy imitation. Additionally, an uncertainty prediction model for the preceding vehicle's behavior is established. By integrating the conVAE model, a personalized reward function is designed, accounting for both safety and comfort and considering the long-term impact of the preceding vehicle's behavior to imitate human-like driving strategies. Finally, the MCQ method optimizes the personalized conVAE strategy, allowing for appropriate exploration beyond the dataset, improving model generalization, and surpassing human performance. Tests on the HighD dataset demonstrate the proposed strategy's superiority in imitating human driver behavior in gap selection, speed variation, and acceleration adjustments. Validation on real-world datasets further confirms its generalization ability and performance, comparable to or better than human drivers.