Evaluating NonDex for Modern Java Ecosystem

NonDex is a testing approach designed to unveil implementation-dependent (ID) flaky tests stemming from in-correctly relying on a deterministic implementation of a Java API with an underdetermined specification, e.g., iterating over elements of a HashSet object. Since the original Nondex work was published in 2016, we have enhanced the tool functionality and expanded its integration with recent Java versions and build tools like Maven and Gradle. This evolution enables Nondex to analyze a broader range of large, open-source Java projects. This paper investigates our updated Nondex on modern Java projects. We identified 734 ID flaky tests in 31 Maven projects and 267 ID flaky tests in 25 Gradle projects. Comparing these findings to prior work, this study highlights an increase for a modern Java project to contain some ID flaky test(s). We also studied the propagation of ID flakiness through project dependencies and fixed a key non-determinism issue in the Gradle build system itself. Our study emphasizes the importance of proactively employing NOndexto detect and fix flaky tests, preventing potential disruptions in ongoing and future projects. We put all our results at https://github.com/NonDexFTW/NonDex-Experiments.