Stability of Machine Learning Predictive Features Under Limited Data

In a world where Machine Learning (ML) is increasingly used to make predictions about critical events, such as health outcomes, it is crucial to ensure decision-makers have access to explainable, consistent, and relevant predictive features. ML prediction relies on perfect data with similar distributions for testing and validation. These results are compared with those of humans, who use more noisy and limited data. Human predictions overcome those limitations by learning from abstractions. This paper addresses these issues by conducting experiments comparing traditional machine learning methods and a previously proposed method that uses data abstractions to learn predictive feature significance. The results indicate that the previously proposed descriptive ML approach maintains higher classification accuracy and ensures the stability of feature selection as data incompleteness increases, becoming valuable under limited data scenarios. It demonstrates the possibility of developing ML capable of automatic decision-making.