Automating Web Application Testing from the Ground Up: Experiences and Lessons Learned in an Industrial Setting

Automating web application testing is a very tricky process due to inherent complexity, dynamic behavior(s) in web pages, differences in the way browsers render the same content (especially on different form factors), and so on. Yet manual testing is not a practical option, and such automation is a must in the interests of effectiveness and efficiency, because of the large number of browsers/devices that users can choose from, and also given the rapid software development cycles of today. This paper discusses our efforts at Varidesk to automate web tests against our main website - which offers many features and content, but is also a true eCommerce site where users from around the globe can purchase a very broad variety of active workspace solutions that we offer. Our solution was developed in-house, from the ground up, and leveraged and extended freely available automation and test libraries such as Selenium WebDriver and NUnit respectively. We talk about the challenges we faced and how we overcame them, as well as provide technical insights on real-world concerns such as managing test brittleness, and integrating the web tests into an existing Continuous Integration and Continuous Deployment (CI/CD) pipeline. Part of the novelty of this paper is that we are also transparent on the rationale behind our decision to build versus buy, and how we managed resources, especially in terms of cost. We also present lessons learned, and encouraged by the success that we have observed, hope that the results will be beneficial to academia and practitioners alike.