A Policy based Deep Reinforcement Learning for Task Offloading and Resource Allocation in Satellite Terrestrial Integrated Internet of Things

Onboard computational resources can be deployed on low earth orbit (LEO) satellites to provide multi-access edge computing (MEC) in a satellite terrestrial integrated Internet of Things (STIOT). User equipments (UEs) lacking computational resources on the ground can choose to offload tasks to a terrestrial gateway that can communicate directly with LEO, and the gateway will consider whether to further offload to LEO and allocate computational and communication resources for tasks. But different task offloading and resource allocation strategies will greatly affect the performance of STIOT and the quality of service for UEs. Therefore, choosing an appropriate policy to minimize the total system delay and energy consumption is a critical issue in STIOT. This problem is a mixed integer nonlinear programming problem and we can't find the optimal analytical solution. So we formulated the system as a constrained markov decision process (CMDP) and adopt the deep reinforcement learning algorithm suitable for STIOT as the offloading strategy, and the Lagrange multiplier algorithm as the resource allocation scheme. Simulation results show that the proposed algorithm can increase the performance of the system by up to 25% compared with benchmark algorithm.