Analysis of the clinical utility of algorithms in the 2009 PhysioNet/Computing in cardiology challenge for the prediction of acute hypotensive episodes

An Acute Hypotensive Episode (AHE) typically indicates an impending life-threatening event for an ICU patient. Therefore, the accurate prediction of AHEs has clear clinical merit and was the focus of the 2009 Challenge. This paper analyzes if the competitors' methods actually provide clinical utility. This analysis was accomplished using the portion of the training set that included records not containing any AHEs. In this training set, the 10 hours prior to the forecast window were analyzed for 15 records using the algorithms described by Chen, Mneimneh, and Langley. When Chen's method was evaluated using a sliding 5-minute window (15 records, 10 hours each), this method generated 884 false alarms. Mneimneh's algorithm produced 131 false alarms over the same 150 hours. Though both Chen and Mneimneh scored 10/10 in Event 1 and 36/40 in Event 2, these results indicate that these methods could never be used practically. Although Langley's algorithm did not perform as favorably in the Challenge, 8/10 in Event 1 and 28/40 in Event 2, it had zero false alarms in the training set used in this study. This indicates that some potential may exist for a relatively simple method to be clinically useful, but performance must be considered both in terms of predictive accuracy as well as low false alarms rates.