ATTEST: A Testing Toolkit for Validating Software Properties

System-level test automation emulates testers' interactions with a System Under Test (SUT) to verify system properties. It is usually achieved through writing scripts in scripting languages, such as Perl or Tel, in order to feeding input to and correlating data from various interfaces of SUT. Test scripts, especially ones requiring thorough results analysis, can easily become complicated and hard to maintain as the software system evolves. ATTEST is a toolkit to address problems in test automation and maintenance. It provides easy-to-use mechanisms for helping testers to write and maintain automated test scripts through describing system behaviors at a high abstract level. It includes a Test Behavior Language (TBL) that uses innovative parameterized patterns to specify and validate trace-based properties abstractly but precisely. A compiler translates TBL specifications into executable scripts. Initial results show that TBL specifications range from 1/2 to 1/5 the size of their script counterparts and can save up to 5 times effort for developing test scripts. TBL demonstrates greater benefit as the complexity of the validation increases.