High- and Low-Frequency Cooperation Based Resource Allocation in Vehicular Edge Computing Via Deep Reinforcement Learning

In vehicular edge computing (VEC) environment, the increasing task offloading requirements from diverse vehicular applications pose significant challenges to the limited and single communication resources. High- and low-frequency cooperation (HL-FC) has the advantages of large capacity, low latency, large coverage capability, and stable communication link during task offloading. However, how to efficiently allocate communication resources for task offloading in the presence of high- and low-frequency communication resources is a challenge. Furthermore, coupled with the allocation of computing resources and the offloading-decision making, the allocation of high- and low-frequency communication resources is even more complex and challenging. To cope with these challenges, in this paper, we investigate the resource allocation scheme under the high- and low-frequency cooperation in VEC. Specifically, to facilitate the processing of latency-sensitive and computation-intensive tasks, a multi-queue model for task caching is first designed to prioritize latency-sensitive workloads, enabling efficient data buffering and processing. Considering vehicle mobility, we then develop the communication model, task migration model, and the computing model. After that, we formulate a long-term average cost optimization problem that jointly optimizes resource expenditure and latency, which is a NP-hard problem. To obtain the optimal strategy, we leverage the Markov decision process (MDP) to model the optimization problem, which is then solved by our proposed twin delayed deep deterministic policy gradient (TD3)-based two-phase resource allocation scheme (TTRAS). Finally, extensive simulations are conducted to assess and validate the effectiveness of the TTRAS.