Hidden Hazards Beneath Cross-Validation Methods in Machine Learning-Based Sleep Apnea Detection

Obstructive sleep apnea (OSA) is a respiratory disorder highly correlated with multiple cardiovascular diseases. In the last two decades, several alternatives have been proposed to palliate the limitations of polysomnography, the current gold standard for OSA diagnosis. Such alternatives were mainly based on the heart rate variability in combination with machine learning (ML) techniques, obtaining promising results. However, the majority of these works used a cross-validation approach for the validation of the proposed methods, and rarely tested them on external sources of newly added data. Hence, some of the most common algorithms found in the state of the art have been evaluated with cross-validation and external validation in this work. The obtained results have raised important concerns on the real performance shown by the typical ML-based OSA detection methods in more realistic scenarios.