Intra-layer model division for Edge Machine Learning Computational Needs

Edge machine learning models on IoT (Internet of Things) devices allow making decisions closer to the environment where the data are gathered. IoT ML Edge devices are associated with computational power limitation. Cloud computing is usually used for its high computational resources. However, high latency, associated costs, and reliability concerns due to the use of the Internet impose limitations on its usage notably for time-sensitive tasks. To address those limitations, this paper proposes an intra-layer model division approach across nearby edge devices, enabling more reliable execution of computationally intensive machine learning models. The execution of some model layers was parallelized according to a load-balancing technique based on the estimated computational power of each device. The experimental results demonstrated that the proposed load-balancing method achieved faster execution compared to cloud computing or single-device execution, albeit influenced by CPU usage and model size.