DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers
2024 IEEE International Conference on Pervasive Computing and Communications (PerCom)(2024)
摘要
Enabling efficient and accurate deep neural network (DNN) inference on
microcontrollers is non-trivial due to the constrained on-chip resources.
Current methodologies primarily focus on compressing larger models yet at the
expense of model accuracy. In this paper, we rethink the problem from the
inverse perspective by constructing small/weak models directly and improving
their accuracy. Thus, we introduce DiTMoS, a novel DNN training and inference
framework with a selector-classifiers architecture, where the selector routes
each input sample to the appropriate classifier for classification. DiTMoS is
grounded on a key insight: a composition of weak models can exhibit high
diversity and the union of them can significantly boost the accuracy upper
bound. To approach the upper bound, DiTMoS introduces three strategies
including diverse training data splitting to increase the classifiers'
diversity, adversarial selector-classifiers training to ensure synergistic
interactions thereby maximizing their complementarity, and heterogeneous
feature aggregation to improve the capacity of classifiers. We further propose
a network slicing technique to alleviate the extra memory overhead incurred by
feature aggregation. We deploy DiTMoS on the Neucleo STM32F767ZI board and
evaluate it based on three time-series datasets for human activity recognition,
keywords spotting, and emotion recognition, respectively. The experiment
results manifest that: (a) DiTMoS achieves up to 13.4
compared to the best baseline; (b) network slicing almost completely eliminates
the memory overhead incurred by feature aggregation with a marginal increase of
latency.
更多查看译文
关键词
embedded machine learning,model diversity,model selection,adversarial training
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要