Pattern Extraction for Behaviours of Multi-Stage Threats via Unsupervised Learning

Detection of multi-stage threats such as Advanced Persistent Threats (APT) is extremely challenging due to their deceptive approaches. Sequential events of threats might look benign when performed individually or from different addresses. We propose a new unsupervised framework to identify patterns and correlations of malicious behaviours by analysing heterogeneous log-files. The framework consists of two main phases of data analysis to extract inner-behaviours of log-files and then the patterns of those behaviours over analysed files. To evaluate the framework we have produced a (publicly available) labelled version of the SotM43 dataset. Our results demonstrate that the framework can (i) efficiently cluster inner-behaviours of log-files with high accuracy and (ii) extract patterns of malicious behaviour and correlations between those patterns from real-world data.