Distributed Joint Congestion Control for V2X Using Multi-Agent Reinforcement Learning

With the advancements of research on autonomous driving, wireless communication between autonomous vehicles is becoming increasingly important so that they can exchange information about their kinematics. Specifically, each vehicle periodically sends Basic Safety Messages (BSM) or Cooperative Awareness Messages (CAM) containing the vehicle’s position, speed, and acceleration to prevent a car crash. However, the Intelligent Transportation System (ITS) band allocated for Vehicle-to-Everything (V2X) communication is typically narrow. As a result, when vehicle traffic density increases, the wireless channel can become congested. Although there are standard congestion control algorithms, they are heuristically designed and have been shown to perform poorly due to the difficulties involved in the joint control. In this paper, we show that Multi-Agent Reinforcement Learning (MARL) can be harnessed to produce an effective joint rate and power control for cellular V2X environment. Our results show that the learned policy can achieve significant Packet Inter-Reception time (PIR) improvement over the standard algorithms while maintaining an appropriate Channel Busy Ratio (CBR). Moreover, the congestion control function can be extracted from the trained network, and is simple enough to be specified in the standard documents.