Machine Learning in Context of IoT/Edge Devices and LoLiPoP-IoT Project*

Machine learning models are traditionally deployed in the cloud or on centralized servers to leverage their computing resources. However, such a deployment may reduce privacy, introduce extra latency, consume more power, etc., and subsequently negatively impact properties of an application that typically runs on a battery-operated device used to communicate via a wireless network. To minimize the negative impact, it is necessary to deploy a model directly to such a device to minimize data transfer energy and run the model closer to the data source and, application and its environment. However, this kind of deployment is a challenging task due to the very limited resources available in such devices and applications. Many people and companies have tackled this challenging problem and proposed different ways and means to solve it. Having defined the problem and our area of interest, the paper provides an overview of representative applications, methods and means, including libraries, frameworks, datasets, devices etc. It then presents a typical deployment process workflow in the context of resource-constrained devices. Finally, it sums representative results for popular resource-constrained devices (e.g., Arduino, ARM Cortex-M, ESP32, nRF5x, Nvidia Jetson, Raspberry Pi) to demonstrate how various phenomena (e.g., model type, setting, quantization) affect model performance (e.g., accuracy, loss), metrics (e.g., ROC AUC, F1 scores) and device performance (e.g., feature and inference processing time, memory usage).