COTS Score: an acceptance methodology for COTS software

Over the last decade the United States Government has significantly increased its use of commercial-off-the-shelf (COTS) software as stand-alone solutions and as components in safety-critical systems. This increased use stems from the realization that pre-existing software products can be a means of lowering development costs, shortening development time, and keeping pace with the changing software market. The Federal government, particularly with regards to safety-critical systems, has found that COTS software is currently not plug-and-play, has significant tradeoffs, and usually contains a "cradle-to-grave" dependence on the software manufacturer. Unfortunately, there is currently no standard "best commercial practice" with regard to the acceptance of COTS software. Ad hoc attempts to apply standards for commercial software acceptance have been based upon subjective criteria and have proven to be imprecise and prone to error. Moreover, acceptability in safety-critical systems generally demands analysis of source code. Many software vendors, however, have chosen to keep all such source code proprietary due to liability and intellectual property concerns. When the source code is not available, there is very little that can be done to ensure the safety, reliability, and integrity of the software. This paper discusses the COTS Score, an approach that aids in determining the acceptability of COTS software. The process involves the application of predictive techniques used for financial credit scoring to the COTS domain. The methodology addresses the issue of acceptability by incorporating both functional and environmental software measures related to reliability, compatibility, certifiability, obsolescence, and life cycle including trade-off analyses. This approach satisfies NASA and ISO 9001 requirements to define an acceptance procedure for COTS software.