A transmission design in dynamic heterogeneous V2V networks through multi-agent deep reinforcement learning

In highly dynamic and heterogeneous vehicular communication networks, it is challenging to efficiently utilize network resources and ensure demanding performance requirements of safety-related applications. This paper investigates machine-learning-assisted transmission design in a typical multi-user vehicle-to-vehicle (V2V) communication scenario. The transmission process proceeds sequentially along the discrete time steps, where several source nodes intend to deliver multiple different types of messages to their respective destinations within the same spectrum. Due to rapid movement of vehicles, real-time acquirement of channel knowledge and central coordination of all transmission actions are in general hard to realize. We consider applying multi-agent deep reinforcement learning (MADRL) to handle this issue. By transforming the transmission design problem into a stochastic game, a multi-agent proximal policy optimization (MAPPO) algorithm under a centralized training and decentralized execution framework is proposed such that each source decides its own transmission message type, power level, and data rate, based on local observations of the environment and feedback, to maximize its energy efficiency. Via simulations we show that our method achieves better performance over conventional methods.