WIVET—Benchmarking Coverage Qualities of Web Crawlers

Web application vulnerability scanners (WAVS) include crawler components to extract all accessible links of tested web pages in order to identify attack entry points and parameters. After extracting links, they perform different types of attacks over each extracted link and try to find out existing vulnerabilities in the tested web application for reporting. A WAVS tool that has a low-quality crawler component would generate false-negative results, since failing to discover existing links would inhibit detection of possible vulnerabilities exposed through these links. Therefore, the coverage quality of its crawler plays a very important role in the success of a WAVS tool. In this paper, we propose a novel method for analyzing and comparing coverage qualities of WAVS crawlers. We developed WIVET (Web Input Vector Extractor Teaser) as a benchmarking tool for analyzing crawler components of WAVS. WIVET evaluates WAVS crawlers based on their extraction capability of 56 target links that are generated statically or dynamically by WIVET's 21 test cases. We explain WIVET's architecture, all WIVET test cases and target links with code examples, integration of WIVET into WAVS development environments and WAVS benchmarking results in detail.