Event Pattern Discovery on IDS Traces of Cloud Services

The value of Intrusion Detection System (IDS) traces is based on being able to meaningfully parse the complex data patterns appearing therein as based on the pre-defined intrusion 'detection' rule sets. As IDS traces monitor large groups of servers, large amounts of network data and also spanning a variety of patterns, efficient analytical approaches are needed to address this big heterogeneous data analysis problem. We believe that using unsupervised learning methods can help to classify data that allows analysts to find out meaningful insights and extract the value of the collected data more precisely and efficiently. This study demonstrates how the technique of growing hierarchical self-organizing maps (GHSOM) can be utilized to facilitate efficient event data analysis. For the collected IDS traces, GHSOM is used to cluster data and reveal the geometric distances between each cluster in a topological space such that the attack signatures for each cluster can be easily identified. The experimental results from a real-world IDS traces show that our proposed approach can efficiently discover several critical attack patterns and significantly reduce the size of IDS trace log which needs to be further analyzed. The proposed approach can help internet security administrators/analysts to conduct network forensics analysis, discover suspicious attack sources, and set up recovery processes to prevent previously unknown security threats such as zero-day attacks.