Scalable Malware Clustering using Multi-Stage Tree Parallelization

Similarity hashing is an important tool for searching and analyzing malware samples which are similar to known malware samples. Several similarity hashing schemes exist in the literature (like ssdeep, TLSH, sdhash). TLSH has been found to be particularly well-suited for finding related malware (and goodware) samples from known malware (and goodware) samples. In particular, TLSH has been shown to be good at finding the different variants of a given malware. Previous work has shown that TLSH hashes can be used to build fast search and clustering techniques which can scale to tens of millions of items. In this paper, we show that previous work can be made to scale to even larger data sizes by doing clustering in stages. A fast clustering algorithm (like k-Means) can be used in multiple stages to obtain clusters at a coarse-level, which can later be processed by other state-of-the-art clustering techniques in parallel to obtain final clusters. We show that such a multi-stage technique can be used to cluster up to 10 million items with 9-12x speedup over just using existing state-of-the-art clustering techniques. We show that the resulting cluster quality obtained by multi-stage clustering is comparable to the cluster quality obtained by existing methods. Moreover, we show how to optimize the cost (dollars spent on the cloud) or latency incurred by multi-stage clustering technique by choosing appropriate values of parameters.