Context-aware, Composable Anomaly Detection in Large-scale Mobile Networks

In a large-scale mobile network, due to the diversity of data characteristics, detection purposes of operation teams, and analytics and machine learning algorithm abilities, building big data anomaly detection pipelines without considering different analytics and team situations may not yield expected quality of analytics, including detection relevancy, performance and quality. This is especially for analytics subjects, such as mobile network zones, of which characteristics are dynamic and contextual. Moreover, due to the lack of labeled data and the high cost of creating labeled data, building anomaly detection analytics models based on (supervised) deep learning or advanced models is even more challenging from various aspects of effort, cost and deployment. In this paper, we present a novel framework that enables anomaly detection through context-aware, composable components to provide efficient detection pipelines suitable for lightweight, resource constrained and geographical operation teams. First, we identify and categorize different types of analytics feature contexts and evaluate existing algorithms suitable for these contexts, mapping anomaly detection algorithms, patterns and configurations for data pre-processing and unsupervised detection tasks in individual analytics functionality. These context-specific pipelines detect anomalies and their relevancy for dynamic analytics subjects such as mobile network zones. Then we develop dynamic configuration and combination techniques for such pipelines to produce highly relevant, multi-context detection of anomalies. Our framework provides flexibility and configurations for team contexts to carry out the anomaly detection in the team’s operations. We will demonstrate our work through real data gathered for a large-scale mobile network covering multiple types of sites with different geographical zones and equipment. We especially focus on district zones and user-defined zones as analytics subjects that must be managed by teams in our experiments.