A/B test tools of native mobile application

When an application developer wants to introduce a new idea to his application, the idea needs to be validated with hypothesis testing in order to prove the treatment variant (application with new idea) will become a better alternative compared with the control variant (base application). In web application development, controlled experiment (known as A/B test) is commonly used as tools for testing a hypothesis. Controlled experiment for native mobile application (hereinafter, mobile application) in this research is created based on controlled experiment in web. We identified three main difficulties in mobile application controlled experiment compared to experiment of web application: element composition, variant delivery, and network connectivity. We propose techniques to overcome the difficulties in controlled experiment for mobile application: splitter generator and plugin architecture + custom build system. This technique is implemented as set of tools for creating variant easily and conducting controlled experiment of mobile application, consists of variant generator, library for tracking experiment execution, a build system, and a monitoring web application. Tools have been tested using two cases. The experiment shows that our tools are able to help developer produce a controlled variant with clearer separation of concern and to conduct controlled experiment easier.