Structure Regularized Traffic Monitoring for Traffic Matrix Estimation and Anomaly Detection by Link-Load Measurements

We consider problems of traffic matrix (TM) estimation and anomaly detection utilizing link-load traffic measurements. Models including structure regularized traffic monitoring (SRTM) and dynamic SRTM (DSRTM) are presented to realize traffic monitoring under static and dynamic routing configurations, respectively. Considering that real traffic data are usually approximately low-rank but exhibit strong spatial and temporal dependencies, we define spatial and temporal regularization matrices based on Moore-Penrose pseudoinverse and Laplacian matrix to structurally regularize the TM variables. Besides, in view of the fact that anomalies in traffic usually happen rarely and last briefly, sparsity-regularization is further implemented on traffic volume anomalies. This enables our models to jointly deal with the traffic monitoring issues of TM estimation and anomaly detection. The SRTM model is designed for static routing configurations, and the online traffic monitoring model DSRTM is presented for dynamic settings. In DSRTM, a forgetting parameter is introduced to incorporate information from both the latest and previous estimations, getting rid of the problem that storing a large amount of traffic data is a huge burden for computers. Furthermore, real-time monitoring enables DSRTM to be applied in scenarios that usually experience nonstationary. Efficient algorithms that are based on accelerated proximal gradient, gradient descent, and block coordinate descent methods are proposed to solve the related SRTM and DSRTM optimization problems, with experiments in synthetic and real networks under both static and dynamic routing configurations, verifying their feasibility and effectiveness.