TinyMM: Multimodal-Multitask Machine Learning on Low-Power MCUs for Smart Glasses

Maintaining a sufficient battery life-time is crucial for an everyday wearable instrumented frame. Therefore, the use of a low-power integrated circuit, such as a microcontroller unit (MCU) is essential. However, MCUs are hardware and software limited (e.g., memory limitations, limited multiprocessing capabilities, real-time constraints, etc.). These limitations restrict the efficient execution of multiple machine learning inferences within the frame. In this regard, we propose in this work, a multimodal multitask machine learning approach for low power micro controllers, called TinyMM. Our proposal has been assessed through a real-world use case for smart glasses, which involves processing data from an accelerometer and a digital microphone to perform three distinct applications: (1) detecting the wearer's activity, (2) identifying the wearer's mode of transportation and (3) locating the wearer's scene. Furthermore, when compared to standard approaches, our proposal showed a 2.7 times reduction in the inference energy and runtime, as well as a 2.1 times reduction in the required storage memory footprint.