A Reinforcement Learning-Driven Scheduler for MPTCP in Heterogeneous Networks

The MPTCP protocol allows packets to be diverted among multiple paths, which can effectively aggregate their bandwidth and thus improve the overall transmission throughput. The packet scheduler is a key component of the MPTCP protocol, which is responsible for link management and data distribution, and how to reasonably allocate packets to multiple paths has been a key topic in multipath transmission research. However, typical schedulers design their scheduling algorithms according to some specific network environments, which have a limited scope of application and cannot adapt to the changing heterogeneous networks. Therefore, we propose a reinforcement learning-based scheduler (RLAS). The RLAS uses a combination of online scheduling and offline training so that it can adapt its scheduling policy according to network changes. We design a heterogeneous network state space, a set of multipaths, and a reward function related to the bandwidth aggregation rate, and then use an Advantage Actor-Critic (A2C) reinforcement learning structure to acquire a target scheduling policy that maximizes the reward. Experiments show that RLAS can adapt well to the changes in the network environment, such as delay, bandwidth, and packet loss rate, and its aggregated throughput is significantly higher than typical schedulers.