Brewing Up Reliability: Espresso Test Generation for Android Apps

The ESPRESSO testing framework for ANDROID has gained popularity among developers as it allows to write concise and reliable VI tests. State-of-the-art tools for automatically testing ANDROID apps, however, tend to produce crash reports rather than human-readable tests, and even if they produce tests these (1) rarely use the ESPRESSO format; (2) are often unreliable due to the volatile nature of widget identifiers; and (3) usually contain no test assertions to serve as regression oracles. While the lack of ESPRESSO support of test generation tools has been addressed by reverse engineering ESPRESSO tests, the other problems remain even with this workaround. In this paper, we therefore introduce a novel ESPREsso-based representation that allows test generators to generate ESPRESSO test cases directly that (1) can reliably identify widgets using clear and concise ESPRESSO selectors, and (2) can check test executions using ESPRESSO assertions. Experiments on 1,035 ANDROID apps demonstrate that the proposed approach generates ESPRESSO tests that are significantly more reliable than reverse engineered tests, and the ESPRESSO assertions of the generated tests are effective at detecting faults in ANDROID apps.

I. Introduction

ESPRESSO [1] is a testing framework that helps developers write concise, reliable, and human-readable ANDROID UI tests, and it is the only UI testing framework with substan-tial adoption amongst app developers [2]. Its popularity is supported by its inclusion in the ANDROID Software Development Kit (SDK), its ability to mitigate flakiness, and the simplicity of the creation and maintenance of tests. Despite this popularity, evidence suggests that ANDROID developers tend to neglect creating automated tests [2], [3].