Addressing an inadequacy in quantitative analysis: examining NMP technology selection

In its early years NASA's New Millennium Program (NMP) applied quantitative analysis methods in making complex decision in selecting technology for flight validation. In 1997, the quantitative approach showed serious signs of weakness in the selection of technologies for DS3. In response to this, the NMP began to explore new methods to select its technologies. To understand this inadequacy in applying quantitative methods and how it has been subsequently addressed, a field study was performed of the current NMP technology selection processes used for ST8. The goal of this study is to capture and analyze the processes and issues surrounding the formation and selection of technology using ethnographic methods. The study found: 1) setting high-level, break-through technological requirements leads to flight concepts that can potentially address a significant application domain, 2) creating and using technology concept areas was key in performing the selection process, 3) producing multiple technology concept areas in necessary to address disparate science interests, 4) promoting competitive choices fosters innovation, while not prejudging the outcome, and 5) obtaining prioritization of flight concepts by the science community balances their interests. These results lead to a deeper finding that the selection processes are not well represented by quantitative analysis methods alone because they do not capture the 1) design understanding and insight and 2) sensemaking and negotiations outcomes needed for the decision process to reach consensus.