HyperAST: Incrementally Mining Large Source Code Repositories

Modern software systems are large, with a project like Chromium reaching more than 30 million lines of code. Analyzing these large-scale projects over multiple versions rapidly becomes very expensive, and creating tools that can work at this scale is a challenge. This paper presents the HyperAST approach, that exploits the locality and redundancy of source code, to maintain thousands of Syntax Tree (AST) versions in memory. In particular, we contribute a programmatic interface to HyperAST that helps define the incremental computation of code metrics and efficient explorations of the fine-grained abstract syntax representation of source code.