Double Deep Q-learning Based Satellite Spectrum/Code Resource Scheduling with Multi-constraint

For multi-user satellite Internet of Things (IoT) systems operating at lower signal-to-noise ratio, spread spectrum techniques are usually used to combat narrowband interference. In addition, the communication performance in the spread spectrum system depends on the anti-jamming ability of the spreading codes (SCs). Therefore, how to design the SCs scheduling strategies under users' requirements and resource constraints has become a crucial problem for satellite IoT systems. In this paper, communication rewards and scheduling delays are introduced as gauges to measure the scheduling performance of the satellite gateway station control center (SGSCC). Specifically, SGSCC must efficiently and effectively allocate limited available SCs over terminal gateways under request at each transmission time slot. The SCs scheduling problem is formulated as a Markov Decision Process (MDP) along with the observed environments composed of resource status and user request status. Then a deep reinforcement learning scheduling algorithm is devised by embedding the idea of Long Short-Term Memory (LSTM) in the standard Double Deep Q-learning (DDQN). Simulation results show that the proposed algorithm can achieve much better performance than traditional algorithms in terms of communication rewards and scheduling delays. Finally, we draw some conclusions.