A Machine Learning Approach for Rate Prediction in Multicast File-stream Distribution Networks

Large-volume scientific data is one of the prominent driving forces behind next generation networking. In particular, Software Defined Network (SDN) makes leveraging path-based network multicast services practically feasible. In our prior work, we have developed a cross-layer architecture for supporting reliable file-streams multicasting over SDN-enabled Layer-2 network, and implemented the architecture for a meteorology data distribution application in atmospheric science. However, it is challenging to determine an optimal rate for this application with the varying type, volume, and quality of meteorological data. In this paper, we propose a Quality of Service (QoS)-driven rate management pipeline to determine the optimal rate based on the input traffic characteristics and performance constraints. Specifically, the pipeline employs a feedtype classifier using Multi-Layer Perception (MLP) to recognize the type of meteorological data and a delay prediction regressor using stacked Long Short-Term Memory (LSTM) to predict per-file delay for the file-streams. Finally, we determine the optimal rate for the given file-streams using the trained regressor. We implement this pipeline to test the real-world file-stream data collected from a trial deployment, and the results show that our regressor outperforms all baselines by selecting the optimal rate in the presence of varying file set sizes.