Astra: Bottom-up Construction of Structured Artifact Repositories

Structured artifact repositories, such as Java's Maven Central Repository, provide developers with the ability to easily locate and manage their programs' external dependencies. Unfortunately, these artifact repositories are often missing popular libraries, dramatically decreasing their usefulness. Our investigation of the scope of the Maven Central Repository showed that it is missing over 60% of externally referenced types in a large collection of open source Java projects. Artifact repositories, such as the Maven Central Repository, are manually curated, and constructed in a top-down manner. This makes expanding their scope, and keeping them up to date, a time consuming process. To address this issue, we present Astra, an algorithm for the automated bottom-up construction of structured artifact repositories. Astra takes as input a collection of unknown library artifacts, such as jar files, and returns a structured artifact repository. The resulting repository contains libraries, which are divided into versions and associated with individual artifacts. This bottom-up construction is accomplished by analyzing the co-occurrence of types between the library artifacts. Astra was implemented for Java as part of the Sourcerer Infrastructure. Our evaluation demonstrates that Astra generates a repository with 77% similarity to the Maven Central Repository when provided the same base artifacts. An examination of the differences between the manually curated and generated repositories indicates that in many cases the generated structure has significant merit.