EdgeEngine: A Thermal-Aware Optimization Framework for Edge Inference

Heterogeneous edge platforms enable the efficient execution of machine learning inference applications. These applications often have a critical constraint (such as meeting a deadline) and an optimization goal (such as minimizing energy consumption). To navigate this space, existing optimization frameworks adjust the platform's frequency configuration for the CPU, the GPU and/or the memory controller. However, existing optimization frameworks have two limitations. First, edge applications are frequently deployed in environments where they are exposed to ambient temperature variations. Second, a recent study has shown that temperature has a significant impact on edge platform characteristics. In this context, today's frequency optimization frameworks (which are thermal-oblivious) select frequency configurations that either violate the application's constraints, or are sub-optimal in terms of the optimization goal. To address these shortcomings, we propose EdgeEngine, a lightweight thermal-aware optimization framework. EdgeEngine monitors the platform's temperature and uses reinforcement learning to adjust the frequency configuration of all underlying platform resources to meet the application's constraints. We find that EdgeEngine meets the application's constraint, and achieves 29% lower energy consumption (up to 2x) and 41% fewer violations compared to existing state-of-the-art thermal-oblivious optimization frameworks.