MDPRP: A Q-Learning Approach for the Joint Control of Beaconing Rate and Transmission Power in VANETs

Vehicular ad-hoc communications rely on periodic broadcast beacons as the basis for most of their safety applications, allowing vehicles to be aware of their surroundings. However, an excessive beaconing load might compromise the proper operation of these crucial applications, especially regarding the exchange of emergency messages. Therefore, congestion control can play an important role. In this article, we propose joint beaconing rate and transmission power control based on policy evaluation. To this end, a Markov Decision Process (MDP) is modeled by making a set of reasonable simplifying assumptions which are resolved using Q-learning techniques. This MDP characterization, denoted as MDPRP (indicating Rate and Power), leverages the trade-off between beaconing rate and transmission power allocation. Moreover, MDPRP operates in a non-cooperative and distributed fashion, without requiring additional information from neighbors, which makes it suitable for use in infrastructureless (ad-hoc) networks. The results obtained reveal that MDPRP not only balances the channel load successfully but also provides positive outcomes in terms of packet delivery ratio. Finally, the robustness of the solution is shown since the algorithm works well even in those cases where none of the assumptions made to derive the MDP model apply.