Model-Based Continuous Integration Testing of Responsiveness of Web Applications

The problem we tackle deals with testing responsiveness of Web applications across different user platforms, which are combinations of browsers, operating systems, devices and device orientation. Because of the variety of such environmental conditions, the behavior of the Web application (the application under test, AUT) can differ widely. Failure modes such as buttons not being visible, feedback boxes located in the wrong position, user interface (UI) components not behaving properly are very common and may affect only some of the platforms. Even when practitioners adopt services such as Browserstack that provide virtualization of user platforms, identification of such defects is very demanding, and as a consequence it is seldom performed in a continuous integration fashion. Unfortunately, layout requirements of UIs tend to change very frequently, and the result is that changes are often not tested well enough. What is needed is a software test automation solution. We tackled this problem by adopting a model-based approach, centered on models that represent the data manipulated by the UI and its behavior. A compiler reads these models, integrates model annotations providing details about UI implementation and about test oracles, and produces Java source code of a test harness. The test harness hides details about the Document Object Model (DOM), about the particular test driver that is used to drive the AUT, and details about the oracles that can be used to check if the AUT satisfies certain properties, such as those dealing with its layout. The test engineer can write high level code for jUnit/TestNG test cases, which can then be run on all the desired platforms, using Selenium as a driver.