Super learning for anomaly detection in cellular networks

The ever-growing population of smartphones connected to mobile networks is changing the cellular traffic ecosystem. The traffic volumes and patterns generated by smartphone apps pose complex challenges to cellular network operators, particularly in terms of detection and diagnosis of network anomalies caused by specific apps. The high-dimensionality of network data provided by current network monitoring systems opens the door to the application of machine learning approaches to improve the detection and classification of network anomalies, but this higher dimensionality comes with an extra data processing overhead. In addition, critical network monitoring applications such as the detection of anomalies require fast mechanisms for on-line analysis of thousands of events per second, as well as efficient techniques for off-line analysis of massive historical data. In this paper we explore the application of big data analytics and big data platforms to the automatic detection of anomalies in mobile networks. We consider ensemble, multi-combined machine learning models to enhance anomaly detection, following a particularly promising model known as Super Learning. Super-learning is an ensemble learning approach which performs asymptotically as well as the best possible weighted combination of multiple learning algorithms, providing a very powerful detection approach. We implement and test different super-learning models on top of Big-DAMA, a big data analytics framework for network monitoring. We test the proposed solution on the detection of traffic anomalies in operational cellular networks, and compare it to other traditional ensemble learning approaches such as bagging and boosting. Results indicate that our approach outperforms traditional ones.