Adaptive Horizon Model Predictive Control with Reinforcement Learning for Highway Local Ramp Metering

Model predictive control is a widely used approach to the problem of highway local ramp metering problem. The main parameter that affects the computational complexity is the prediction horizon. Considering the performance sensitivity to the prediction horizon length, this paper proposes a reinforcement learning (RL) based adaptive horizon model predictive control (AHMPC). In the RL-AHMPC, the prediction horizon is adjusted based on an RL agent according to the system states. The RL agent learns the adaptive horizon policy by continuously interacting with the environment. Meanwhile, an effective MPC controller is designed to handle the constraints and objectives in the ramp metering problem, and provide optimal control sequences. Simulation results show that the novel RL-AHMPC method can improve traffic efficiency and keep the computational cost at a low level. This is due to the combination of the optimal control sequences from the MPC controller and the intelligent adaptive horizon generated by the RL agent.