Leveraging Temporal Patterns: Automated Augmentation to Create Temporal Early Exit Networks for Efficient Edge AI

In embedded systems, efficient deep learning solutions are crucial to balance accuracy and resource constraints. Early Exit Neural Networks offer a promising solution, but their manual configuration and optimization hinder widespread adoption. To overcome this hurdle, we propose a novel, fully automated flow that transforms traditional neural networks into Temporal Decision Early Exit models, optimizing their temporal decision mechanism for efficient inference. Temporal decisions adapt the inference workload by monitoring changes in early predictions over time. This enables them to significantly improve latency and efficiency when operating on streaming data. Our approach enables significant latency reductions without the need for expert knowledge while maintaining prediction quality, making it ideal for resource-constrained embedded systems. We demonstrate the effectiveness of our approach on three representative tasks: ECG classification, wake word detection, and video-based human presence detection. Our results show latency reductions of up to 28.6%, 42.5%, and 24.5% compared to the traditional inference execution, respectively, with minimal accuracy loss. By enabling broader adoption of Temporal Decision Early Exit Neural Networks, our method has the potential to transform the field of embedded deep learning and unlock new possibilities for edge AI.