Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN
arxiv(2023)
摘要
Large pretrained models, coupled with fine-tuning, are slowly becoming
established as the dominant architecture in machine learning. Even though these
models offer impressive performance, their practical application is often
limited by the prohibitive amount of resources required for every inference.
Early-exiting dynamic neural networks (EDNN) circumvent this issue by allowing
a model to make some of its predictions from intermediate layers (i.e.,
early-exit). Training an EDNN architecture is challenging as it consists of two
intertwined components: the gating mechanism (GM) that controls early-exiting
decisions and the intermediate inference modules (IMs) that perform inference
from intermediate representations. As a result, most existing approaches rely
on thresholding confidence metrics for the gating mechanism and strive to
improve the underlying backbone network and the inference modules. Although
successful, this approach has two fundamental shortcomings: 1) the GMs and the
IMs are decoupled during training, leading to a train-test mismatch; and 2) the
thresholding gating mechanism introduces a positive bias into the predictive
probabilities, making it difficult to readily extract uncertainty information.
We propose a novel architecture that connects these two modules. This leads to
significant performance improvements on classification datasets and enables
better uncertainty characterization capabilities.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要