Toward Diverse Tiny-Model Selection for Microcontrollers

Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is challenging due to their constrained on-chip resources. Existing approaches mainly focus on compressing larger models, often compromising model accuracy as a trade-off. In this paper, we rethink the problem from the inverse perspective by directly constructing small/weak models, then enhancing their accuracy. Thus, we propose DiTMoS, a novel DNN training and inference framework featuring a selector-classifiers architecture, where the selector routes each input sample to the appropriate classifier for classification. DiTMoS is built on a key insight: a combination of weak models can exhibit high diversity and the union of them can significantly raise the upper bound of overall accuracy. To approach the upper bound, DiTMoS introduces three strategies including diverse training data splitting to enhance the classifiers’ diversity, adversarial selector-classifiers training to ensure synergistic interactions thereby maximizing their complementarity, and heterogeneous feature aggregation to improve the capacity of classifiers. We further design a network slicing technique to eliminate the extra memory consumption incurred by feature aggregation. We deploy DiTMoS on the Nucleo STM32F767ZI board and evaluate its performance across three time-series datasets for human activity recognition, keyword spotting, and emotion recognition tasks. The experimental results show that: (a) DiTMoS improves accuracy by up to 13.4% compared to the best baseline; (b) network slicing successfully eliminates the memory overhead introduced by feature aggregation, with only a minimal increase in latency.