Emergency Message Broadcast Mechanism in Vehicular Ad-Hoc Networks Based on Reinforcement Learning With Contention Estimation

In vehicular ad-hoc networks (VANETs), ensuring passenger safety requires fast and reliable emergency message broadcasts. The current communication standard for messaging in VANETs is IEEE 802.11p. As IEEE 802.11p allows carrier-sense multiple access with collision avoidance (CSMA/CA) in the media access control (MAC) layer. A large contention window (
$CW$
) value will increase delay, whereas a small
$CW$
value will increase the probability of collision. Therefore, adaptive regulation of the
$CW$
value is needed to achieve high reliability and low delay in VANETs, in accordance with variations in the environment. However, the traditional MAC protocol cannot achieve the aforementioned requirements. Reinforcement learning (RL) emphasizes the selection of optimal action according to observations of the environment to achieve optimal system performance. In this study, a Q-learning (QL) RL algorithm based on IEEE 802.11p was used to achieve the requirements of adaptive broadcasting. Adaptive broadcasting was achieved based on a reward definition of high reliability and low delay for the QL algorithm. In this approach, the learning state is the
$CW$
size, the system sets up a Q-table using RL, and the optimal action is based on the maximum Q-value. The
$CW$
size can be provided with adaptive self-regulation by RL, providing high reliability and low delay for the broadcast of emergency messages. We also compared our proposed scheme to other QL-based MAC protocols in VANETs by performing simulations and demonstrated that it can achieve high reliability and low delay for the broadcast of emergency messages.