LENTA: Longitudinal Exploration for Network Traffic Analysis From Passive Data

In this paper, we present longitudinal exploration for network traffic analysis (LENTA), a system that supports the network analysts in the identification of traffic generated by services and applications running on the Web. In the case of URLs observed in operative network, LENTA simplifies the analyst's job by letting her observe few hundreds of clusters instead of the original hundred thousands of single URLs. We implement a self-learning methodology, where the system grows its knowledge, which is used in turn to automatically associate traffic to previously observed services, and identify new traffic generated by possibly suspicious applications. This approach lets the analysts easily observe changes in network traffic, identify new services, and unexpected activities. We follow a data-driven approach and run LENTA on traces collected both in ISP networks and directly on hosts via proxies. We analyze traffic in batches of 24-h worth of traffic. Big data solutions are used to enable horizontal scalability and meet performance requirements. We show that LENTA allows the analyst to clearly understand which services are running on their network, possibly highlighting malicious traffic and changes over time, greatly simplifying the view and understanding of the network traffic.