Neural Network Decomposition and Distribution on Multiple Microcontrollers

The deployment of Neural Network (NN) models on low-power and resource-constrained devices represents a critical bottleneck in the design of intelligent and autonomous Internet of Things (IoT) systems due to the aggressive computational and memory constraints. In this paper, we introduce a novel design methodology based on a distributed approach, which aims at automatically partitioning the execution of a NN over multiple heterogeneous tiny devices. Such a methodology is formalized as an optimization problem where either the inference latency is minimized or the throughput is maximized, within the devices’ memory and computing capabilities. The methodology is evaluated over several NN architectures and microcontrollers (MCUs) by using three algorithms, namely Full Search (FS), Dichotomic Search (DS) and Branch-and-Bound (B&B). The obtained results showed that the B&B outperformed the others since, in all the experiments, it was able to find the optimal solution in the lowest number of computing steps. With this work, we aim at enabling novel Machine Learning (ML) solutions which offer low latency, autonomy and high energy efficiency.