Towards a Library of Deep Neural Networks for Experimenting with on-Device Training on Microcontrollers

Despite the limited computing capacity of microcontrollers, there is a growing interest in training neural networks on these devices for tiny machine learning applications. The traditional approach is to train models off-device on powerful machines, but this limits the model's ability on the microcontroller to adapt to changing data. A limitation for further research on on-device training is the lack of open source deep neural network implementations for the training on microcontroller boards. In this paper, we present the design and implementation of a library in C++ to enable neural network training and inference on microcontroller boards. We conduct a functional validation of the implementation by comparing a Convolutional Neural Network (CNN) and autoencoder model defined with our library with a reference implementation. We then integrate the library into a machine learning application and flash it on an Arduino Portenta board to demonstrate its usability for on-device training. The developed library is open source and is flexible for defining different neural network architectures. Researchers and developers can use the given implementation as a tool for building new trainable machine learning applications, but can also extend it with recent model optimization techniques for tiny nodes from the research in this field.