Contextual Transformer-based Node Embedding for Vulnerability Detection using Graph Learning

Automated source code vulnerability detection using code graphs has seen major improvements in recent years, however one critical, but oft-overlooked, element of this problem is producing embeddings for graph nodes. Before graph-based classifiers can be used for vulnerability detection, the nodes in the graph must first be given vector representations. Graphlearning models propagate information from these embeddings through the graph before classification, and so the initial states of these embeddings are vital for all subsequent learning. While a variety of solutions to this problem have been proposed in existing literature, this is typically not the focus of these works. We propose a novel node embedding strategy for graph-based vulnerability discovery, which takes advantage of richly-learned information about the code contained in each node. We also implement and test several existing node embedding strategies, comparing them to each other and our new strategy under a standard graph-learning architecture. We find that our strategy outperforms existing methods by 10.47-50.70%.