Applying System Readiness Levels to Cost Estimates – A Case Study Part 2

Systems and systems of systems (SoS) continue to become more common, interrelated and complex. To understand how cost and schedule is impacted, System Readiness Level (SRL) techniques can be applied to forecast program cost and schedule to initial and full operational capabilities (IOC & FOC). A prior paper applied SRL techniques to validate the method against an actual NASA flagship program, the James Webb Space Telescope (JWST). It was demonstrated that a Technology Readiness Assessment (TRA) applied to a large complex system provides optimistic results with the actual Technology Readiness Level (TRL) being much lower when including system integrations. The result shown on JWST was unexpected cost growth and schedule delays. The application of SRL techniques provided a data point that adds merit and objectiveness to the method when estimating cost and schedule of weapon systems. Additionally, it isolates specific sub systems needing additional maturity during program development. This “Part 2” effort looks specifically at long running Department of Defense (DoD) weapon system development programs that have been chronicled in the Government Accountability Office's (GAOs) annual review of selected weapons systems and related reports. We will augment GAO's knowledge point framework with SRL methods to add fidelity that highlight areas needing additional maturity to bring the system up to the required standards. This paper assesses reports from 2009 to 2019 where we selected two programs of the 82 reviewed to analyze and contrast using their historical reports. Demonstrating that System Readiness Levels (SRLs) techniques work require rigor and objective assessment. Applying heuristic cost and schedule analysis reinforces capabilities for pre-development cost estimates and schedule completions within the acquisition lifecycle. Up to now, theoretical and analytical approaches are in the literature. This and the prior paper build on actual program cost and schedule history through program development. Augmenting SRL forecasting techniques to aforementioned approaches using actual program data adds to its validity. Evaluations from multiple programs begin to provide a basis for the null hypothesis and demonstrate a statistically significant approach that can influence cost and schedule estimating process improvement. While more analysis is needed, this is a first step. Moreover, cost and schedule realism at the start of “Program of Record” can be defined with higher confidence. Future research will add more program data points to enhance the robustness of the method; then model enhancements can be confidently made adding to a population defining cost and schedule drivers within the systems and SoSs.