Forensic System Verification

Once a system is built, stakeholders are faced with the task of determining the degree to which the system as a whole and major subsystems meet requirements. However, in real-world embedded distributed systems, it can be impossible, impractical, or too costly to gather enough field test data to make these evaluations in a straight-forward manner using traditional system testing alone. Instead, we need a method of verifying requirements compliance that uses the evidence available while minimizing the chances of wrong or misleading conclusions. This paper introduces the novel forensic system verification (FSV) method, which combines modeling, data mining, analytic extrapolation, and goal modeling. It augments human inferences with automated reasoning about requirement satisfaction and confidence propagation. The FSV Method is evaluated on a case study performed using data from a recent medium-scale field trial of a complex distributed embedded system.