NetMiner: Identifying Failure-Inducing Patterns in the Logs Generated by Network Monitoring Software

To ensure uninterrupted network communication among multiple sources, the network administrator needs to do real-time monitoring of network logs. Due to its huge volume and streaming nature, it is almost impossible to do a manual analysis of the logs generated by different network monitoring software. Moreover, these logs may contain a limited amount of data regarding the identity, location, capacity, etc. of a user due to privacy issues. Therefore, we propose an automated system named NetMiner for performing log analysis that requires only insensitive data such as the network’s current status and messages related to that status. We collect recent 30 days logs using the network monitoring tool PRTG (Paessler Router Traffic Grapher) and perform data analysis by leveraging a data mining technique called Fpmax. Our proposed technique NetMiner can identify patterns that may lead to failure and warn the network administrator about upcoming failures. Our experimental results show that NetMiner is both effective and efficient in identifying failure-inducing patterns in logs generated by network monitoring software.