Justifying Software Testing in the 21st Century

The role of testing as a software verification activity was directly challenged at a recent practitioners' conference. The speaker's points were that testing cannot usefully demonstrate software dependability, that it is over-costly, and that there is a case to be made for omitting it from the (high-integrity) software development process altogether. This paper attempts to contrast verification techniques including testing against a variety of criteria. The principal rival techniques considered include two relatively new ones: advanced defect detection and formal verification. In both cases there are grounds for saying that their use will lead to 'better' source code, but there is still ultimately a reliance on the compiler as the means to turn source code into executable code. Since compilers are generally flawed themselves it is argued that testing is the only 'ultimate guarantee' of correct execution. The option of bypassing verification altogether by means of code generation is also considered, but makes no difference to the overall argument