Graph Convolutional Network Based Multi-Objective Meta-Deep Q-Learning for Eco-Routing

Route selection can greatly affect vehicle fuel consumption and emissions. Finding the most fuel/energy-efficient route is known as the eco-routing problem. Existing eco-routing solutions do not effectively consider the critical traffic signal information and rely on fuel consumption models that may not be sufficiently accurate. To address the eco-routing problem in a signalized traffic network, this paper proposes a graph convolutional network based multi-objective meta-deep Q-learning (GM
$^{\bm{2}}$
DQL) method. The problem is formulated as dynamic multi-objective Markov decision processes (MOMDP) and is tackled through deep reinforcement learning and meta-learning. We identify that graph convolutional network (GCN) is an efficient and suitable feature representation for a signalized traffic network. GM
$^{\bm{2}}$
DQL can explore the optimal routes with respect to drivers’ different preferences on saving fuel and travel time. Through GM
$^{\bm{2}}$
DQL, the agent is trained under a series of learning environments that are characterized by historical vehicle trajectories, fuel consumption data, and traffic signal data in the remote data center. The vehicle requesting eco-routing service can download the model that represents the action value function of the historical dynamic driving conditions. The model in the vehicle can quickly adapt to the most recent driving condition through online one-shot learning and predict the optimal eco-routes for the subsequent unseen driving conditions of the signalized traffic network. Extensive proof-of-concept experiments validate that GM
$^{\bm{2}}$
DQL can effectively discover optimal eco-routes. It saves up to 71% travel time and 62% fuel, compared to the conventional shortest-path routing strategy that is widely used in navigation systems.