Automated Accessibility Testing of Mobile Apps

It is important to make mobile apps accessible, so as not to exclude users with common disabilities such as blindness, low vision, or color blindness. Even when developers are aware of these accessibility needs, the lack of tool support makes the development and assessment of accessible apps challenging. Some accessibility properties can be checked statically, but user interface widgets are often created dynamically and are not amenable to static checking. Some accessibility checking frameworks analyze accessibility properties at runtime, but have to rely on existing thorough test suites. In this paper, we introduce the idea of using automated test generation to explore the accessibility of mobile apps. We present the MATE tool (Mobile Accessibility Testing), which automatically explores apps while applying different checks for accessibility issues related to visual impairment. For each issue, MATE generates a detailed report that supports the developer in fixing the issue. Experiments on a sample of 73 apps demonstrate that MATE detects more basic accessibility problems than static analysis, and many additional types of accessibility problems that cannot be detected statically at all. Comparison with existing accessibility testing frameworks demonstrates that the independence of an existing test suite leads to the identification of many more accessibility problems. Even when enabling Android's assistive features like contrast enhancement, MATE can still find many accessibility issues.

I. Introduction

An estimated 15% of the world population experience some form of disability and face barriers in regular life [41], including access to information and communication technology (ICT). The ICT infrastructure is increasingly shifting towards applications for mobile devices (“apps”), where accessibility thus is becoming a more pressing concern. For example, Fig. 1 shows screenshots of the Google Play app, used on Android devices to access the official app store. While this is a well designed and pleasant user interface, users with visual impairment may struggle with the low contrast ratio of the category labels at the top of the left hand screen. Moreover, users with physical and motor skill issues might struggle to interact with the highlighted buttons since they are too small.