BobGAT: Towards Inferring Software Bill of Behavior with Pre-Trained Graph Attention Networks

In recent years, a variety of Graph Neural Network (GNN) and Natural Language Processing (NLP) techniques have been proposed for binary analysis tasks including assembly embedding generation and binary similarity. However, a majority of these techniques were developed and tested on rather restrictive datasets or tasks, generally limited to one compiler, a limited number of compiler flags, and a small or artificially augmented dataset ill-suited for the semantic understanding of binaries required to generate a software bill of behavior. To that end, we have scraped a new programming competition dataset an order of magnitude larger than previous largest datasets, as well as designed a novel GNN architecture tailor-made for Control Flow Graph (CFG) analysis. We achieve over 92% accuracy on a complex 8000+ class programming problem classification task and perform extensive ablation studies showing the increased efficacy our model architectures provide and the necessity of proper dataset construction and deduplication for generalizability. We pre-train our models on an embedding task designed to improve our models’ semantic understanding of binaries and showcase their ability to group similar binaries together by behavior. Finally, we train our model as-is on a popular malware benchmark dataset and achieve near state-of-the-art performance with little effort: 99.23% accuracy on the Microsoft Malware Classification Challenge (Big 2015) dataset. Our data preparation tools are open-sourced, well-documented and tested, and pip-installable.