Automated Testing of Science Instrument Flight Software

This paper describes an approach to automated testing of science instrument flight software using scripted commanding, telemetry capture, and intelligent telemetry verification. The most difficult piece of this approach, telemetry verification, is accomplished with a JPL-developed software package, HKCheck, which takes user- defined packet formats, error conditions, and goals, and verifies that binary telemetry meets the specified conditions. Packet formats are defined using an ASCII file, easily created in any text editor. Test-specific and context-specific error conditions are expressed using a simple C-like notation, enabling large amounts of telemetry from normal flight operation or system testing to be rapidly scanned for anomalous telemetry. Goals, like waypoints, are similarly expressed in simple notation, and allow telemetry analysts or test conductors to look for specific events in the telemetry stream. The power and flexibility of the HKCheck error and goal specification language enables context-dependent analysis of telemetry, and rapid, repeatable analysis of test results, encouraging more frequent testing, and reducing the risk of human error in test analysis. Use of HKCheck on Mars Climate Sounder enabled full regression testing of flight software every month or two, rather than just on major deliveries, giving developers confidence that software was working correctly and uncovering potential bugs earlier in the development cycle than standard methods allow. It has since been used, with extensions, on other flight projects as well.