Early warning look ahead metrics: The Percent Milestone Backlog metric

This paper describes the purpose, development methodology, and utility of the Percent Milestone Backlog metric that was jointly developed by National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) and The Aerospace Corporation to monitor the execution performance of GSFC flight projects that are under development. All complex development projects experience delays and corresponding backlogs of their project control milestones during their acquisition lifecycles. As part of a larger set of Early Warning Look Ahead metrics that were developed to monitor GSFC projects' cost and schedule performance, the Percent Milestone Backlog metric is particularly revealing, and has utility as a stand-alone execution performance monitoring tool that measures performance independently of Earned Value Management (EVM), and provides insights of project performance relative to historical successful projects. Monthly milestone backlog data were collected on a large collection of historical NASA flight projects. The backlog data were converted to a percent backlog relative to the planned milestone completion performance, and were then normalized to cover the period of acquisition from Preliminary Design Review (PDR) to Launch Readiness Date (LRD). From this normalized dataset, a “region of normality” was discovered that depicts historical percent milestone backlog versus time. This region describes the maximum and minimum percent backlog observed by the historical projects at any stage of their acquisition lifecycle beginning at PDR. Given this region, the metric then plots the percent backlog performance of a project that is being monitored as a function of time from PDR to the present. Those projects whose percent backlog falls within the “region of normality” as described by the historical projects are considered “healthy,” and behaving normally despite their backlog. On the other hand, those projects whose percent backlog falls outside the “region of normality” are considered to be at risk of a schedule overrun, and worthy of further intervention. Furthermore, the “region of normality” is subdivided into “shades of normality,” providing an indication that a currently “healthy” project may be headed for trouble. The Percent Milestone Backlog metric provides actionable insight (independent of EVM) to GSFC management on the milestone execution performance of GSFC flight project acquisitions. But the method could easily be extended to any kind of commodity that can extract similar performance data. For NASA, this method could be applied to other centers such as Marshall Space Flight Center or Kennedy Space Center. The method could also be applied to other agencies and organizations. GSFC management has found the Percent Milestone Backlog metric to be an important tool in their management oversight toolkit.