RLSS: A Reinforcement Learning Scheme for HD Map Data Source Selection in Vehicular NDN

In the autonomous driving era, high-definition (HD) maps are an essential building block to enable fine-grained environmental perception, precise localization, and path planning. However, with rich multidimensional information, the size of HD map data is huge and cannot be stored onboard, where the dynamic map data need to be distributed in real time via vehicular networks and how to design the distribution mechanism (i.e., determining the data source for requests) becomes crucial. For the end-to-end communication protocols (i.e., TCP/IP), the main limitation is the vehicle mobility and high dynamic of the network topology, which can degrade the transmission performance dramatically. Therefore, in this article, we propose a reinforcement learning-based data source selection scheme, named RLSS, for efficient HD map distribution in vehicular named data networking (NDN) scenarios, which aims at seeking the best data source (roadside infrastructures or nearby vehicles) in accordance with the map data requests. Specifically, in RLSS, we adopt a deep reinforcement learning-based architecture to learn a neural network as an agent to make the decision of data source selection, which can work online after offline training based on historical selection action performance. In addition, to solve the “cold start” problem for a new vehicle, we propose a model aggregation algorithm and weight update approach to learn the model parameters from its nearby vehicles, which can guarantee the performance while saving the communication cost. Finally, we implement RLSS in NS-3 by adopting the tools of the ndnSIM, SUMO, and Gym. Extensive simulations demonstrate that RLSS can significantly improve the transmission performance in terms of delay, throughput, and packet loss when compared with state-of-the-art data source selection schemes.