Collective Deep Network Matrix Factorization for Large-Scale Modeling of the Internet Backbones

We propose a framework for large-scale modeling of the Internet backbone, specifically the backbone of Research and Education Networks (RENs). The traffic on the RENs behave different from the public Wide Area Networks (public WANs) because RENs primarily carry traffic between participating research and education institutions. Although the traffic on public WAN backbones have been analyzed extensively, the same cannot be said about the RENs. The core theoretical model in our proposed framework extends the concept of Deep Matrix Factorization (DMF) to create a model which we call the Collective Deep Network Matrix Factorization (CDNMF). CDNMF is designed to capture and model the underlying hierarchical spatio-temporal nature of the REN backbone traffic. To develop our framework, we studied a large collection of network traffic collected from the core router nodes on the Internet2 backbone (an REN in the US). The collected dataset spans a three-month period with each day consisting of billions of records. The results using our proposed framework on the collected dataset is a proof-of-concept and shows the feasibility of our approach in modeling the traffic on the REN backbones. We present results of our proposed approaches and show that they are useful for large-scale traffic modeling scenarios.