SCRUBD: Smart Contracts Reentrancy and Unhandled Exceptions Vulnerability Dataset

Smart Contracts (SCs) handle transactions in the Ethereum blockchain worth millions of United States dollars, making them a lucrative target for attackers seeking to exploit vulnerabilities and steal funds. The Ethereum community has developed a rich set of tools to detect vulnerabilities in SCs, including reentrancy (RE) and unhandled exceptions (UX). A dataset of SCs labeled with vulnerabilities is needed to evaluate the tools’ efficacy. Existing SC datasets with labeled vulnerabilities have limitations, such as covering only a limited range of vulnerability scenarios and containing incorrect labels. As a result, there is a lack of a standardized dataset to compare the performances of these tools. Our dataset, SCRUBD, aims to fill this gap. SCRUBD is a dataset of real-world SCs and synthesized SCs labeled with RE and UX vulnerabilities. The real-world SC dataset is labeled through crowdsourcing, followed by manual inspection by an experienced SC programmer, and covers both RE and UX vulnerabilities. On the other hand, the synthesized dataset is carefully crafted to cover various RE scenarios only. Using SCRUBD, we compared the performance of six popular vulnerability detection tools. Based on our study, we found that Slither outperforms other tools on a crowdsourced dataset in detecting RE vulnerabilities, while Sailfish outperforms other tools on a manually synthesized dataset for detecting RE. For UX vulnerabilities, Slither outperforms all other tools.