Electric Vehicle Charging Guidance Strategy Based on Hierarchical Multi-Agent Deep Reinforcement Learning

To address effectively the stochastic charging demand of electric vehicles, this paper proposes a charging guidance strategy for electric vehicles based on hierarchical multi-agent deep reinforcement learning. This strategy refines the charging guidance task into a double-layer finite Markov decision process for the complex feature information in the coupled system of electric vehicles, charging stations, and transportation networks. In this architecture, the upper network adopts a centralized learning and decentralized execution strategy, and each electric vehicle is regarded as an independent agent. These agents cooperate in diverse incentives and competitive environments to jointly recommend the best charging station locations. At the same time, the action decision outputs of the upper network are integrated into the state considerations of the lower multi-agents. The lower network adopts the same solution method as the upper network, and the lower multi-agent plans the optimal driving paths based on the latest EV and traffic network states after receiving the upper decision. Further, a multi-agent hybrid Q learning algorithm is introduced to solve the problem, and constraints are imposed on the agent rewards to guide the multi-agent learning. Finally, simulation results validate that the proposed strategy enhances electric vehicle charging efficiency and promotes efficient road network operation.