Discretizing Technical Documentation for End-to-End Traceability Tests

This paper describes a technique by which English prose protocol standards were transcribed into individually testable assertions and traced from original protocol specifications, to lists of requirements, to models and test cases, and finally into test logs and network captures. Annotating statements to make them stand alone, the handling of optional behavior, and using the requirements to guide discovery of missing information from the standard is also described. The approach allows us to have both a fast feedback loop for debugging the specification of the protocol and a way to estimate the coverage degree of the generated tests with respect to requirements. We discuss tool support developed specifically for the approach and exemplify with excerpts from the application of this technique in the process of testing hundreds of Microsoft's open protocol specifications.