Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles

Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.