SimML Framework: Monte Carlo Simulation of Statistical Machine Learning Algorithms for IoT Prognostic Applications

Advanced statistical machine learning (ML) algorithms are being developed, trained, tuned, optimized, and validated for real-time prognostics for internet-of-things (IoT) applications in the fields of manufacturing, transportation, and utilities. For such applications, we have achieved greatest prognostic success with ML algorithms from a class of pattern recognition known as nonlinear, nonparametric regression. To intercompare candidate ML algorithmics to identify the "best" algorithms for IoT prognostic applications, we use three quantitative performance metrics: false alarm probability (FAP), missed alarm probability (MAP), and overhead compute cost (CC) for real-time surveillance. This paper presents a comprehensive framework, SimML, for systematic parametric evaluation of statistical ML algorithmics for IoT prognostic applications. SimML evaluates quantitative FAP, MAP, and CC performance as a parametric function of input signals' degree of cross-correlation, signal-to-noise ratio, number of input signals, sampling rates for the input signals, and number of training vectors selected for training. Output from SimML is provided in the form of 3D response surfaces for the performance metrics that are essential for comparing candidate ML algorithms in precise, quantitative terms.