Reinforcement Learning Based Variable Speed Limit Control for Mixed Traffic Flows

Today's urban mobility requires results for resolving increasingly complex demands on the traffic management system. Hence, the main problem is to achieve a satisfactory level of service for urban motorways as part of the urban traffic network. In addition, with the introduction of Connected and Autonomous Vehicles (CAVs), additional challenges for modern control systems arise. This study focuses on the Variable Speed Limit (VSL) based on Q-Learning with CAVs as actuators in the control loop. The Q-Learning algorithm is combined with the two-step Temporal Difference target to increase the effectiveness of the algorithm for learning the VSL control policy for mixed traffic flows. Different CAV penetration rates are analyzed, and the results are compared with a rule-based VSL and the no control case. The obtained results show that Q-Learning based VSL can learn the control policy and improve the Total Travel Time and Mean Travel Time for different CAV penetration rates. The results are most apparent in the case of low CAV penetration rates. There is also an indication that the increase of the CAV penetration rate reduces the need for separate VSL control.