Heterogeneous Markov Decision Process Model for Joint Resource Allocation and Task Scheduling in Network Slicing Enabled Internet of Vehicles

In the Internet of Vehicles (IoV) with network slicing functions, both the inter-slice resource allocation and the intra-slice task scheduling have received extensive attention. However, the joint optimization of these two problems has not been fully studied. This letter focuses on the complexity issue of the joint resource allocation and task scheduling problem in the IoV with both ultra-reliable low-latency communication (URLLC) and enhanced mobile broadband (eMBB) slices. A heterogeneous Markov decision process (HMDP) which models the resource allocation process and the task scheduling process in two different layers is proposed. Benefiting from the two-layer structure of the HMDP, the size of action space is significantly reduced, and the asynchronous decisions of different sub-MDPs are enabled. The corresponding layered deep reinforcement learning (DRL) architecture is also designed to solve the HMDP-based optimization problem. Simulation results show that the layered DRL-based algorithm outperforms other methods.