Fine-Grained HTTP Web Traffic Analysis Based on Large-Scale Mobile Datasets

Mobile web traffic analytics helps mobile network operators to understand subscriber behaviors and network characteristics. With the increasing prevalence of mobile devices, people are prone to access Internet by mobile devices. Meanwhile, a large amount of applications and services are being moved to the mobile web. These changes have resulted in the elevated complexity of mobile web traffic. Consequently, original web traffic models require an adjustment, and network understanding methods also need to update. In this paper, we proposed a stream algorithm to identify user click requests, and reconstructed user-browser interactions by leveraging Spark Streaming framework. The proposed algorithm was tested by massive real HTTP traffic records, which were captured from a cellular core network by high-performance monitoring devices. A statistical analysis was made on the reconstructed data set, and the overall characteristics of mobile web traffic were presented as well. Finally, major improvements in mobile web traffic models were obtained, and the key factors affecting web performance were found. We believe that our models and findings can be useful for mobile network operators to exhaustively understand the mobile web traffic and effectively analyze subscriber behaviors.