A Leading Cruise Controller for Autonomous Vehicles in Mixed Autonomy Based on Preference-Based Reinforcement Learning

Previous studies on car-following controllers for autonomous vehicles (AVs) in mixed traffic have a narrow focus on maximizing the AV's utility, neglecting the utility of the entire traffic flow. This leads to self-centered AVs that may not be beneficial to surrounding vehicles. Thus, this study aims to develop a leading cruise controller for AVs that considers not only the AV's behaviors, but also the behaviors of both the lead human-driven vehicle (LHDV) and the following human-driven vehicle (FHDV). To achieve this, the study uses real-world data from the Waymo Open Dataset to approximate the behaviors of human-driven vehicles (HDVs) through an inverse reinforcement learning (IRL) approach. The study then proposes a preference-based soft actor-critic (PbSAC) algorithm to optimize the speed of AVs in a three-vehicle car- following scenario, while also considering safety, efficiency, and string stability for both AV and FHDV in the reward function. To further improve the control algorithm, the study develops a preference-adjusting module that adaptively updates the weights of the reward function based on expert evaluation. Experimental results show that the proposed algorithm can significantly improve safety, efficiency, and string stability for both AV and FHDV.