Memory-Efficient Neural Network Deployment Methodology for Low-RAM Microcontrollers Using Quantization and Layer-Wise Model Partitioning

This paper introduces a memory-efficient methodology for deploying neural networks (NNs) on low-RAM microcontrollers. The work utilizes a compact multilayer perceptron (MLP) with 50 neurons and 716 trainable parameters, achieving a classification accuracy of 97.22% on the MNIST-10 dataset. The trained MLP is split into separate, fully connected submodels to conserve RAM, each representing a specific layer. The submodels are sequentially executed in the order they appear in the original MLP, with only the active submodel in RAM at any given time, drastically reducing the RAM requirements. These submodels utilize the weights and biases of the pre-trained MLP, eliminating the need for additional training. Following this methodology, the trained model only requires a maximum of 1.3kB of RAM (along with the whole infrastructure) to process one MNIST image. In contrast, initially, the model parameters alone would have a cost 2.8 kB of RAM. The model parameters are integer quantized to save on flash memory and converted to floating-point during runtime with minimal accuracy loss. The RAM based execution also enables low-latency inference, primarily for batch processing tasks, and can be adapted to deploy a wider range of pretrained neural networks on low-RAM microcontrollers. A proofof-concept is demonstrated on an Atmega328p Arduino board with two kB RAM and 32 kB flash memory.