Multi-Objective Proximal Policy Optimization for Digital Twin-Assisted Computing Offloading in Internet of Vehicles

To cope with high latency and communication overhead in the Internet of Vehicles (IoV), digital twin (DT) technology is employed in mobile edge computing (MEC) to provide real-time monitoring and analysis. In this paper, we investigate the DT-assisted IoV edge computing problem with the goal of minimizing the long-term task offloading cost in terms of the offloading latency, energy consumption, and cloud computing cost. Then, we propose a multi-objective reinforcement learning (RL)-based DT-assisted vehicular edge computing (VEC) solution, which utilizes a large amount of real-time mapping data from DT to train the neural network in RL. Through combining the multi-objective evolutionary algorithm based on decomposition (MOEA/D) and proximal policy optimization (PPO) algorithm, the agent can make task offloading decisions simultaneously under the unknown dynamics of the vehicular environment. Simulation results show the superiority of the proposed DT-assisted solution compared to other single-objective solutions and the counterparts without DT in terms of striking a long-term trade-off among multiple objectives.