Automated software robustness testing - static and adaptive test case design methods

Testing is essential in the development of any software system. Testing is required to assess a system's functionality and quality of operation in its final environment. This is especially of importance for systems being assembled from many self-contained software components. In this article, we focus on automated testing of software component robustness, which is a component's ability to handle invalid input data or environmental conditions. We describe how large numbers of test cases can effectively and automatically be generated from small sets of test values. However, there is a great demand on ways to efficiently reduce this mass of test cases as actually executing them on a data processing machine would be too time consuming and expensive. We discuss static analytic methods for test case reduction and some of the disadvantages they bring. Finally a more intelligent and efficient approach is introduced, the Adaptive Test Procedure for Software Robustness Testing developed at ABB Corporate Research in Ladenburg. Along with these discussions the need for intelligent test approaches is illustrated by the Ballista methodology for automated robustness testing of software component interfaces. An object-oriented approach based on parameter data types rather than component functionality essentially eliminates the need for function-specific test scaffolding.