Lightweight Adaptive Mixture of Neural and N-gram Language Models.
arXiv: Computation and Language(2018)
摘要
It is often the case that the best performing language model is an ensemble of a neural language model with n-grams. In this work, we propose a method to improve how these two models are combined. By using a small network which predicts the mixture weight between the two models, we adapt their relative importance at each time step. Because the gating network is small, it trains quickly on small amounts of held out data, and does not add overhead at scoring time. Our experiments carried out on the One Billion Word benchmark show a significant improvement over the state of the art ensemble without retraining of the basic modules.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络