Improved DDPG Based Two-Timescale Multi-Dimensional Resource Allocation for Multi-Access Edge Computing Networks

Due to the dependence of task processing on the task-related models and databases, edge service caching and multi-access edge computing (MEC) are tightly coupled. The service caching placement, offloading and resource allocation have become the key to guarantee the latency and computing requirements of tasks. However, much of the existing work is based on unrealistic assumptions that caching and other resources can be scheduled simultaneously, which lead to frequent cache switching and high system overhead. To address this, we propose a two-timescale multi-dimensional resource optimization scheme based on a Markov decision process (MDP), which jointly optimizes the long-term service caching and short-term task offloading, computing, and bandwidth resource allocation to minimize the long-term system delay and cache cost. To accommodate the two-timescale characteristics, we propose a centralized dual-actor deep deterministic policy gradient (DDPG) algorithm, where there is a dual-actor network producing both long-term and short-term resource management decisions, and a centralized critic network directing joint actions. Such a structure strengthens the learning process, enabling the dual-actor network to generate different timescale decisions that align with consistent objectives. Simulation results show that the proposed algorithm can reduce delay and cache cost compared to the existing scheme.