Systematically Testing and Diagnosing Responsiveness for Android Apps

App responsiveness is the most intuitive interpretation of app performance from user's perspective. Traditional performance profilers only focus on one kind of program activities (e.g., CPU profiling), while the cause for slow responsiveness is diverse or even due to the joint effect of multiple kinds. Also, various test configurations, such as device hardware and wireless connectivity can have dramatic impact on particular program activities and indirectly affect app responsiveness. Conventional mobile testing lacks mechanisms to reveal configuration-sensitive bugs. In this paper, we propose AppSPIN, a tool to automatically diagnose app responsiveness bugs and systematically explore configuration-sensitive bugs. AppSPIN instruments the app to collect program events and UI responsiveness. The instrumented app is exercised with automated monkey testers and AppSPIN correlates excessive and lengthy program events with bad responsiveness detected at runtime. The diagnosis process also synthesizes the major resource bottleneck for the app. After one test run, AppSPIN automatically alters the test configuration to with most bottlenecked resource to further explore responsiveness bugs happened only with particular test configurations. Our preliminary experiments with 30 real-world apps show that AppSPIN can detect 123 responsiveness bugs and successfully diagnose the cause for 87% cases, within an average of 15-minute test time. Also with altered test configurations, AppSPIN uncovers a notable number of new bugs within four extra test runs.