Requirements discovery during the testing of safety-critical software

This paper describes the role of requirements discovery during the testing of a safety-critical software system. Analysis of problem reports generated by the integration and system testing of an embedded, safety-critical software system identified four common mechanisms for requirements discovery and resolution during testing: (1) Incomplete requirements, resolved by changes to the software, (2) Unexpected requirements interactions, resolved by changes to the operational procedures, (3) Requirements confusion by the testers, resolved by changes to the documentation, and (4) Requirements confusion by the testers, resolved by a determination that no change was needed The experience reported here confirms that requirements discovery during testing is frequently due to communication difficulties and subtle interface issues. The results also suggest that "false positive" problem reports from testing (in which the software behaves correctly but unexpectedly) provide a rich source of requirements information that can be used to reduce operational anomalies in critical systems.