Automated Neural and On-Device Learning for Micro Controllers

mW power consumption envelope, and below, is a pillar of the TinyML community, which aggregates contributions on algorithms, tools, software, and hardware for machine learning at the edge. In the tinyML context, this paper presents the state-of-the-art review of different approaches for Neural Architecture Search targeting resource constrained devices such as microcontrollers (MCU). As well as the implementations of On-Device Learning techniques for them. On the former, part of the paper reviews different approaches of Neural Architecture Search which consider the hardware constraints into the search mechanism and how they have been able to design a tiny neural network with low memory and computational requirements which can be deployed on cheap and off-the-shelf MCU. On the latter, the paper reviews the approaches and advanced solutions for On-Device Learning with MCU to address the concept drift and cope with the accuracy drop on real time data. Moreover, this paper also introduces Extreme Learning Machines (ELM) for feature extraction and some preliminary results using Restricted Coulomb Energy neural network for On-Device Classification Learning which is highly storage conservative and more suitable for tiny, embedded devices.