Assurance Equations: A Cost and Criticality Model for Optimizing Quality Assurance Surveillance

The cost of quality vs cost of failure correction has been a long-running topic of discussion within the Aerospace community. It leads directly to concepts of “risk tolerance”, and risk-based decision-making. It would be valuable if there was a way to compute the optimal investment in customer-executed quality assurance activities using defect significance with respect to performance objectives, the activities' defect detection effectiveness, and the cost-penalty for late discovery of impactful defects. This optimization is particularly of interest to projects whose budget constraints significantly limit their risk management options. The cost to fix defects (i.e., failure correction) escalates as the project matures. There have been studies attempting to determine the relative cost of fixing defects discovered during various phases of a project life cycle with important implications, all of which suggest growth factors are large. The commonly referred to 1:10:100 rule represents a cost multiplier for repair/rework across the Design to Fab to Test hardware development phases. Cost premiums for Quality Assurance (QA) activities also accumulate when they are treated as mandatory (due to schedule drag) or are performed later than their assigned phase. This paper describes the modeling of development phase dependencies in the conduct of typical customer-executed quality assurance activities. Our initial modeling encompasses: •Distinct phases of the production lifecycle •Multiple kinds of defects, each with some a-priori likelihood of being present •Each defect's impact on performance objectives for a type of hardware •The cost and efficacy of assurance techniques at detecting such defects •The costs of fixing those defects detected in a given phase of the production lifecycle The model captures assurance activities' abilities to detect defects. Upon detection it is assumed that the defect is immediately fixed. Defects that “escape” detection by some activity may thereafter be detected by a later activity, but by then the cost of fixing the defect may have escalated. Defects are related to the performance objectives they would detract from, were those defects to remain present in the operating system. We have constructed and are exploring a model that relates the importance of hardware system elements to mission objectives, the impact of types of defects on those hardware types, the cost of customer-executed assurance activities (i.e., supplier controls) and their effectiveness towards reducing an impactful quality escape, and the cost of defect correction across the production lifecycle. We describe the approach taken to select the key model aspects, why they are relevant to our NASA mission, and our efforts to populate it with relevant and contemporary data. We use a notional example to illustrate model design and function.