Reliability growth and the caveats of averaging: A Centaur case study

Spacecraft reliability modeling is plagued by data scarcity and lack of data applicability. Systems tend to be one-of-a-kind, and observed failures tend to be the result of systemic defects or human errors, instead of component failures. The result is too often a gap between two extreme estimating approaches: probabilistic risk assessments (PRA) that are component-based lead to optimistic estimates by ignoring system-level failure modes; while history-based failure frequencies can lead to pessimistic estimates by neglecting non-homogeneity (between vehicles and vehicle configurations), reliability growth, and improvements in design. The problem of non-homogeneity is often considered solved once a system has a sufficiently long history. But in reality, rarely can tens of launches be considered samples of the same probability distribution. Launch vehicles undergo design changes in their history; more accurate estimates of reliability need to account for the risk introduced by design changes and for two types of reliability growth: growth of a given system via systematic tracking, assessing, and correction of the causes of failure uncovered in flights; and general technological or knowledge growth over subsequent generations of the system. Using the interesting history of the Centaur upper stage as an example, this paper proposes a pragmatic approach for the estimation of reliability growth over successive flights and configurations, which is applicable to any system with a history of several tens of flights. First considering the Centaur history as a single family, the paper compares the total success frequency to the `instantaneous' success frequency over intervals of increasing flight number. This analysis shows that as a result of the reliability growth experienced by Centaur, the total success frequency underestimates the risk of the first Centaur launches by a factor of almost 10, and overestimates the risk of the last Centaur launches by a factor of more than 3. But a closer analysis of Centaur history reveals that a number of failures were the results of design changes, as the stage design was improved or adapted for flight on new launch vehicle models. Understanding the risk introduced by design changes is important in the use of historical failure data as a surrogate for new systems. The second part of the paper shows that the `interval' growth curve of the Centaur family is the average of distinct growth curves for each configuration. Over a given flight interval, the average success frequency can underestimate the risk of the newest generation of Centaur, and overestimate that of the older operating Centaur, by a factor of 2 to 5. The net result is that after almost 200 flights, the most reliable Centaur presented 10 times less risk than suggested by the total failure frequency, and 100 times less risk than the initial launches. Thus the `mature' reliability was close to typical values generated by some bottom-up PRAs; but it was reached only after a long flight experience and the character of the residual failures is different. The authors hope that the practical approach presented in this paper can be of use to the industry in bridging the gap between forecasts based solely on historical failure frequencies and the results of component-based PRAs; and that it can foster a better understanding of the uncertainty bounds associated with various estimation methods, generally improving the relevance of reliability estimates to the problems faced by launch program decision makers.