JATTACK: Java JIT Testing using Template Programs

We present JATTACK, a framework that enables compiler testing using templates. JATTACK allows compiler developers to write a template program that describes a set of concrete programs to be used to test compilers. Such a template-based approach leverages developers' intuition on testing compilers, by allowing developers to write a template program in the host programming language (Java), which contains a basic program structure while provides an opportunity to express variants of specific language constructs in holes. Each hole, written in a domain-specific language embedded in the host language, is used to construct an extended abstract syntax tree (eAST), which defines the search space of a language construct, e.g., a set of numbers, expressions, statements, etc. JATTACK executes the template program to fill every hole by randomly choosing a number, expression, or statement within the search space defined by the hole, and it generates concrete programs with all holes filled. We used JATTACK to test Java just-in-time (JIT) compilers, and we have found seven critical bugs in Oracle JDK JIT compiler. Oracle developers confirmed and fixed all seven bugs, five of which were previously unknown, including two CVEs (Common Vulnerabilities and Exposures). JATTACK blends developers' intuition via templates with random testing to detect bugs in compilers. The demo video for JATTACK can be found at https://www.youtube.com/watchvmeCFPxucqk4