Reinforcement Learning Based Variable Speed Limit Control for Mixed Traffic Flows Using Speed Transition Matrices for State Estimation

The ever-increasing growth of the car industry and the demand for personal vehicles have put current traffic management systems and infrastructure to strain. The enlarged number of vehicles in traffic flows often creates congestion due to the increased demand to use the existing road capacity. This is especially evident in urban areas that consist of urban roads and urban motorways. Increasing the capacity by building additional infrastructure is not always a feasible solution. Thus, approaches derived from Intelligent Transportation Systems are frequently used to increase the level of service, especially on urban motorways. The development of Connected and Autonomous Vehicles (CAVs) creates additional challenges and opportunities for the traffic management system to cope with. In this study, the Variable Speed Limit (VSL) based on Q-Learning (QL) with CAVs as actuators and mobile sensors combined with Speed Transition Matrices (STMs) for state estimation named STM-QL-VSL is developed and analyzed. Varying traffic scenarios with different CAV penetration rates are analyzed, including the comparison of motorway configuration with one and two applicable VSL zones. The developed STM-QL-VSL algorithm managed to learn the control policy for each tested scenario and improve measured macroscopic traffic parameters such as Total Time Spent and Mean Travel Time.