Avoiding System Failures with Event Interval Probability – 737 MAX Case Study

Summary & ConclusionsA newly developed data analysis methodology drawing upon statistics, probability and reliability theory and Monte Carlo simulation uses event interval probabilities to recognize real reliability degradation in a system with random variation in the data. The analysis method uniquely allows a complete probabilistic analysis with a dataset as small as one; therefore, it is useful for systems requiring high reliability where system failures must be few. It has been shown capable of avoiding failures such as those leading to the Boeing 737 MAX crashes with the 346 fatalities and tens of billions of dollars in economic loss [1]. The purpose of this paper is to encourage use of the analysis method in the future – not only in commercial aviation but other applications. The 737 MAX datasets provide an opportunity to illustrate both the method and value of the data analysis.Updated datasets for the Boeing 737 MAX crash events and precursor events are analyzed and results presented. Aspects of the analysis methodology applied to the 737 MAX have been presented in various papers since 2020 [1], [2], [3], using the best data and estimates that were assembled at that time. Here we consolidate the applicable elements of the analysis method with results in one paper, using current best publicly available raw data for the analysis. The event interval probability (EIP) analysis input data used is that which was available at the time of the crash or precursor. A precursor event is a relatively minor event that can be an early warning of an impending major event such as a crash. The 737 MAX precursor is "sustained stick-shaker activation".The unreliability of the 737 MAX is identified upon the first crash with a probability of 0.02202 that the event could be random variation of a fleet that is as reliable as the worldwide fleet. This p-value rejects a null hypothesis and unreliability distributions – departures to crash distributions – allows risk to be quantified. The distributions are formed with Monte Carlo simulations to solve a math problem – not computer simulation in the normal sense of evaluating a model. The probability of a crash within the nine days to an air worthiness directive incurred a risk with expected value of approximately 23.0 fatalities.Upon the second crash, the probability of the events occurring in such a short number of departures by random chance is 0.00084. With the null hypothesis rejected, the risk associated with continued operation of the unreliable fleet for three days before grounding, considering all the fleet flying, is about 7.9 fatalities. This high risk is due to a currently unrecognized long left tail of the unreliability distribution. Because of high fatality risk within even short intervals, the risk of uncorrected continuation in-service should be quantified immediately upon any serious event with low p-values.The first sustained stick-shaker activation precursor occurred on October 28, 2018. Input data for this precursor is calculated from publicly available sources. EIP analysis could recognize the event as a statistical outlier and provide an opportunity to avoid even the first crash. The probability of the precursor is 0.0235 before a Bayesian-like prior probability is applied. The final probability adjusted for the prior information is 0.0148. This low p-value would alert to the event interval being a statistical outlier signaling need for investigation. This would provide another opportunity to recognize a possible reliability issue, ground the problem aircraft, and investigate. If action on this precursor had been taken, it is possible to have avoided the first crash. An appreciation for the importance of precursors is a prerequisite to their analysis.