A Journey from Manual Testing to Automated Test Generation in an Industry Project

Test automation is essential in fast-paced agile development environments. The main goal is to speed up test execution cycles and to reduce the effort involved in running tests manually. We took test automation one step further and applied test generation to a GUI-based application developed in a large industry project. The paper describes the transition from manual exploratory testing to automated GUI test generation. Key les-sons to be learned are: (1) the test automation pyramid proposed for agile development tends to underestimate the need for high-level GUI testing, (2) automated test generation does not reduce test effort but shifts it to writing test adapters and checks, and (3) the effort for analyzing results produced by generated tests limits the practical application of automated test generation. The report describes the successful application of test generation in a real-world industry project, but it also highlights several open issues to be addressed by future research on test automation.