Temporal Decisions: Leveraging Temporal Correlation for Efficient Decisions in Early Exit Neural Networks
arxiv(2024)
摘要
Deep Learning is becoming increasingly relevant in Embedded and
Internet-of-things applications. However, deploying models on embedded devices
poses a challenge due to their resource limitations. This can impact the
model's inference accuracy and latency. One potential solution are Early Exit
Neural Networks, which adjust model depth dynamically through additional
classifiers attached between their hidden layers. However, the real-time
termination decision mechanism is critical for the system's efficiency,
latency, and sustained accuracy.
This paper introduces Difference Detection and Temporal Patience as decision
mechanisms for Early Exit Neural Networks. They leverage the temporal
correlation present in sensor data streams to efficiently terminate the
inference. We evaluate their effectiveness in health monitoring, image
classification, and wake-word detection tasks. Our novel contributions were
able to reduce the computational footprint compared to established decision
mechanisms significantly while maintaining higher accuracy scores. We achieved
a reduction of mean operations per inference by up to 80
accuracy levels within 5
These findings highlight the importance of considering temporal correlation
in sensor data to improve the termination decision.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要