Probabilistic Risk Assessment for Acceptance Testing

This case study discusses a situation where both the government and contractor are working towards an acceptance protocol that would increase production throughput while properly balancing risk between the two parties. There are multiple competing risk factors in this instance: the risk to the Warfighter risk of accepting underperforming weapons (consumer's risk), the risk of turning away quality weapons (producer's risk), and the risk associated with reduced throughput and not meeting weapon fleet requirements.This case describes, in particular, the dispersion requirements of the weapon. The weapon should have a 90 percent chance of ten out of ten rounds in a shot-grouping landing in a four-inch circle out to 50 meters consistently throughout the weapon's lifetime. [1] The only way to determine this in a binary sense would be to shoot ten targets, and that only for a point estimate. More targets would be required for any level of statistical confidence in the results. In order to convert this to a continuous response, weapon dispersion was taken to be a bivariate normal distribution where the standard deviation describes the spread of the individual shots. Data from Production Verification Testing (PVT) was used to calculate shot group standard deviations and link them to a probability of being covered by a four inch disk via Generalized Linear Models (GLM). Initially, the contractor utilized an in-line quality inspection approach in order to determine production quality for the dispersion performance. This 100% inspection method was particularly time consuming in that each weapon had to be removed from the production line, brought to a firing range, and a ten round shot grouping had to be fired to test the weapon's dispersion. The use of this inefficient quality inspection method meant that throughput could not be increased to meet the fleet requirements of the Joint Services. A traditional approach would require a variables sampling plan in the form of MIL STD 414 (ANSI/ASQC Z1.9). [2] In this case study, a probabilistic approach was taken in order to model future acceptance outcomes and make sampling trade-offs. In order to accomplish this, a resampling technique that was able to develop probability distributions for acceptance based on existing historical data was utilized. Shot group data from previous 100% testing Lot Acceptance Tests (LAT) was used to generate a database of by weapon standard deviations from which to simulate various sampling procedures and make risk based decisions on implementing sampling procedures to increase throughput while still managing risk.