EdgeEngine: A Thermal-Aware Optimization Framework for Edge Inference

Heterogeneous edge platforms enable the efficient execution of ma-chine learning inference applications. These applications often have a critical constraint (such as meeting a deadline) and an optimization goal (such as minimizing energy consumption). To navigate this space, existing optimization frameworks adjust the platform's frequency configuration for the CPU, the GPU and/or the mem-ory controller. However, existing optimization frameworks have two limitations. First, edge applications are frequently deployed in environments where they are exposed to ambient temperature variations. Second, a recent study has shown that temperature has a significant impact on edge platform characteristics. In this context, today's frequency optimization frameworks (which are thermal-oblivious) select frequency configurations that either vio-late the application's constraints, or are sub-optimal in terms of the optimization goal. To address these shortcomings, we propose EdgeEngine, a light-weight thermal-aware optimization framework. EdgeEngine mon-itors the platform's temperature and uses reinforcement learning to adjust the frequency configuration of all underlying plat-form resources to meet the application's constraints. We find that EdgeEngine meets the application's constraint, and achieves 29% lower energy consumption (up to 2x) and 41% fewer violations compared to existing state-of-the-art thermal-oblivious optimization frameworks.