Scheduling Stochastic Real-Time D2D Communications

To meet the real-time requirements of device-to-device (D2D) communications, we study the scheduling problems of real-time D2D communications to handle the challenges introduced by the stochastic packet arrivals, packet losses, and heterogeneous deadlines. In this paper, we first model the scheduling problem by the Markov Decision Process (MDP), and then propose an e-optimal scheduling algorithm based on this model. Considering the curse of dimensionality of the MDP-based model, we further model the scheduling problem by the restless multi-armed bandit model. Based on the model, we prove the indexability of the scheduling problem and derive the closed-form Whittle's index. According to the closed-form index, we thus propose an index-based scheduling algorithm that has very low time complexity. Finally, numerous simulations demonstrate the effectiveness of the proposed algorithms. The e-optimal scheduling algorithm converges very fast to optimal solutions with small-sized networks and small e. The index-based scheduling algorithm outperforms the compared algorithms in terms of the total discounted reward.