An Adaptive Q-Value Adjustment-Based Learning Model for Reliable Vehicle-to-UAV Computation Offloading

Unmanned Air Vehicle (UAV) has been widely used as the flying edge server to support ground vehicles’ Onboard-Unit (OBU) applications. In this work, we address the challenges of training an adaptive learning model which can be deployed on distributed energy-limited UAVs for making highly-reliable low-latency vehicle-to-UAV (V2U) computation offloading. Firstly, we formulate a two-objective mixed integer programming (MIP) problem for optimizing the energy consumption and offloading utility under the robust reliability constraints. The generalized Chebyshev inequality is applied to transform the chance constraints, and then, the minimum transmission power which satisfies the reliability threshold under the worst case is derived. Then, we decompose the primal problem into the IP subproblem while guaranteeing the Pareto optimality. An adaptive Q-value adjustment based deep reinforcement learning (ADRL) model is proposed, which calculates the expected return in theoretic via the heuristic algorithm, and uses it to replace the Q-value from the target network. The replacement is conducted at an adaptive frequency for saving training time and improving learning results. Comprehensive studies demonstrate the advantages of the proposed ADRL in improving the offloading utility, energy efficiency and convergence rate, when comparing with other classical DRL models and optimization algorithms.