QoS Prediction for Component Services in 5 G via Graph-based Deep Reinforcement Learning

The accurate prediction of Quality of Service (QoS) in terms of response time, packet loss rate, latency and throughput for component services is essential for 5 G to fulfill specific Service Level Agreements (SLAs). However, most current efforts failed to fully exploit the time-varying mobility features of users and parallel iteration multi-rules to perform QoS prediction for component services, incurring poor prediction accuracy. Therefore, in this paper, we are devoted to accurate QoS Prediction of Component Services (QPCS) for 5 G via Graph-based Deep Reinforcement Learning (GDRL) to tackle this issue. Towards this end, the QoS prediction is modeled as GDRL-based QoS tensor factorization by designing a Spatio-Temporal-Recurrent-based Graph Attention Network (STR-GAT) and introducing it into Deep Deterministic Policy Gradient (DDPG) to factorize QoS tensor with multiple available rules in parallel. Specifically, a low-rank QoS tensor and an adjacency tensor are established, which include partial QoS observations of component services in each Base Station (BS), along with some missing elements, and evolving spatial information of users across these BSs, respectively. Then, the novel STR-GAT is designed by introducing spatio-temporal relations into conventional GAT to fully derive the mobility features of users to explore potential actions, while the derivative DDPG is adopted to perform tensor factorization with multiple available rules in parallel. Furthermore, the action smoothing and hierarchical-based replay buffer with priority-based and random sampling are designed and introduced into DDPG to stabilize training process and accelerate model convergence. Experimental simulation results on real-world datasets validate the superiorities of QPCS compared with the state-of-the-art approaches in predicting the QoS of component services in 5 G.