Delay-Windowed and Jitter-Sensitive Dynamic Routing using Safe Advantage Actor-Critic

One efficient means to accommodate the proliferation of real-time Internet applications is dynamic QoS (Quality of Service) routing, which, however, remains a challenge if it is required to guarantee punctual packet delivery meeting stringent end-to-end delay and jitter demands. To address this difficulty, we adopt the paradigm of reinforcement learning and present a novel jitter-driven dynamic routing algorithm based on safe advantage actor-critic (A2C). To retain regularity and jittersensitivity of the routes, we augment A2C with a delay-windowed safe route exploration strategy and meanwhile reshape the reward function to take into account the kurtosis of end-to-end delays, which together can facilitate safe, precise jitter control. Our experimental results demonstrate that improvements on the network performance are indeed enabled with consideration of the delay window and delay kurtosis in routing.