Advanced Self-Improving Ramp Metering Algorithm based on Multi-Agent Deep Reinforcement Learning

We proposed a novel ramp metering algorithm embedding multi-agent deep reinforcement learning (DRL) techniques, based on the data of loop detectors. A multi-agent DRL framework is adopted to generate proper ramp metering scheme for each ramp meter in real time to improve the operation efficiency of urban freeway with less investment. A simulation platform is developed to simplify the implementation and training of the algorithm. A set of simulation experiments - encompassing both single and multi-ramp scenarios with various traffic demand profiles - are conducted. Comparing with the state-of-the-practice ramp metering methods, the simulation results demonstrate that the proposed DRL-based algorithm outperforms in a comprehensive evaluation index considering mainstream speed at bottleneck and queue size on ramp. The method presents robustness, scalability, and the capability of further improvement by online learning during implementation.