DeepFlow: Towards Network-Wide Ingress Traffic Prediction Using Machine Learning At Large Scale

Describing incoming web traffic - as seen from large eyeball networks, i.e. ingress traffic - and estimating it into the future, are necessary operations for network service providers who need to efficiently organize the essential tasks of more dynamic network planning and capacity management. For that a network-wide view on ingress traffic processes and their predictions is necessary. We propose DeepFlow, a system that processes complete ingress traffic flow data on a carrier scale and produces forecasts for all traffic flows using Machine Learning techniques. The viability of DeepFlow is shown by comparing different prediction methods on recent and real-world data that covers three years from 2016 to 2019. We use neural and non-neural methods that produce accurate results in predicting the three largest ingress traffic flows. Furthermore, we investigate the case where the traffic time series data has high volatility. We also use a VAR model to generate directed acyclic graphs to get insights into the relationships between the different ASes. DeepFlow is currently deployed in a lab environment of a large European service provider. The initial evaluation results demonstrate the feasibility to realize system-wide, continuous, near real-time and configurable traffic flow prediction at large scale.