Control Strategy of Hard Shoulder Running at Intelligent Freeway Merging Areas Based on Deep Reinforcement Learning

The hard shoulder running (HSR) strategy has proven to be an effective measure to alleviate freeway traffic congestion. This paper proposes a smart HSR strategy based on deep reinforcement learning (DRL), referred to as DRL-S-HSR, designed to improve traffic efficiency in freeway merging areas within a connected environment. We propose the relevant assumptions and specific implementation rules for scenarios involving hard shoulder running in merging areas, along with the lane-changing motivations and safety conditions for connected and autonomous vehicles (CAVs) when temporarily using the hard shoulder. Furthermore, we also propose a vehicle collision avoidance method in merging areas based on the ECR-DQN algorithm, to avoid conflicts between vehicles on the hard shoulder and those on the on-ramp. A three-lane freeway in China is used as a case study. The study analyzes the overall traffic impact in terms of travel time, congestion patterns, carbon emissions, and driving comfort. Two key traffic conditions are also tested, including different CAV penetration rates and ramp inflows. The results show that the DRL-S-HSR strategy significantly alleviates traffic congestion on the freeway mainline under medium to high-density traffic flow conditions. In terms of carbon emissions, exhaust emissions throughout the entire traffic scenario are greatly reduced. Additionally, this strategy improves the driving comfort of merging areas. Overall, the proposed strategy demonstrates significant control effect and good applicability when the CAV penetration rate is between 10% and 20%, and the ramp inflow is less than 600 veh/h, providing a reference for future traffic management measures.