Multi-Agent Assessment With QoS Enhancement for HD Map Updates in a Vehicular Network and Multi-Service Environment

Reinforcement Learning (RL) algorithms have been increasingly applied to tackle the complex challenges of offloading in vehicular ad hoc networks (VANETs), particularly in high-density and high-mobility scenarios where network congestion leads to significant latency issues. These challenges are further exacerbated by the introduction of low-latency applications, such as high-definition (HD) Maps, which are compromised in the current IEEE 802.11p standard due to their low-priority classification. In our previous work, we developed a novel coverage-aware Q-learning algorithm using a single-agent approach to address these concerns. However, a key question remains: how does this solution perform when scaled to a larger, more complex environment using a multi-agent system? To address this, our current study evaluates the scalability and effectiveness of the previously developed single-agent Q-learning solution within a distributed multi-agent environment. This multi-agent approach is designed to enhance network performance by leveraging a smaller state and action space across multiple agents. We conduct extensive evaluations through various test cases, considering factors such as reward functions for individual and overall network performance, the number of agents, and comparisons between centralized and distributed learning. The experimental results show that our proposed multi-agent solution significantly reduces time latency in voice, video, HD Map, and best-effort cases by 40.4%, 36%, 43%, and 12%, respectively, compared to the single-agent approach. These findings demonstrate the potential of our solution to effectively manage the challenges of VANETs in dynamic and large-scale environments.