Computation Offloading and Resource Allocation in Satellite-Terrestrial Integrated Networks: A Deep Reinforcement Learning Approach

Satellite networks can enhance global network coverage without geographical restrictions. By applying edge computing paradigm to satellite networks, they can provide communication and computation services for Internet of Remote Things (IoRT) mobile devices (IMDs) at any time. Thus, in 6G mobile systems, satellite networks serve as a complement to terrestrial networks. This paper places its emphasis on satellite-terrestrial integrated networks (STINs). Computation offloading and resource allocation (CORA) play a crucial role in STINs given the limited communication and computation resources of satellites. In this paper, we investigate the CORA to minimize system energy consumption while ensuring the latency tolerance via jointly optimizing the offloading decision and the allocation of radio and computation resources. Taking into account the dynamic characteristics of network conditions, the CORA problem is formulated as a Markov decision process (MDP). Subsequently, an algorithm based on twin delayed deep deterministic policy gradient (TD3) is designed to automatically determine the optimal decisions. Finally, the convergence and superiority of the proposed algorithm in terms of energy efficiency are evaluated through extensive simulation experiments.