Web Application Testing with Customized Test Requirements - An Experimental Comparison Study

Test suite reduction uses test requirement coverage to determine if the reduced test suite maintains the original suite's requirement coverage. Based on observations from our previous experimental studies on test suite reduction, we believe there is a need for customized test requirements for Web applications. In this paper, we examine usage-based customized test requirements for the test suite reduction problem in Web application testing. We conduct an extensive experimental study to evaluate the tradeoffs between five classes of customized requirements with respect to reduced test suite size, program coverage and fault detection effectiveness. Our results show that the reduced suites' program coverage and fault detection effectiveness increases with the context or data associated with the reduction requirement. Based on our experimental results, we provide guidance to testers on the most useful test requirement for Web applications in general and provide intuition on factors testers need to consider when selecting test requirements