Dynamic Prioritization in Regression Testing

Although used extensively in industry, regression testing is challenging from both a process management as well as a resource management perspective. In literature, proposed test case prioritization techniques assume a constant pool of test cases with non-changing coverage during the regression testing process, and therefore they work with a fixed, prioritized test suite. However, in practice, test cases and their coverage metrics may change during regression testing due to modifications of software artefacts (e.g. due to bug fixing). For example, modifying obsolete test cases or source code may change the coverage metrics during the process. This may lead to some changes in test case priorities. Dealing with manual tests cases, scheduling test case execution in shared environments and other constraints in practice may cause the same effect. In this paper, we highlight these challenges in industrial regression testing and propose a paradigm called Dynamic Prioritization, which uses in-process events and the most up-to-date test suite to re-order test cases.