Enhancing The Recurrent Neural Networks With Positional Gates For Sentence Representation
NEURAL INFORMATION PROCESSING (ICONIP 2018), PT I(2018)
摘要
The recurrent neural networks (RNN) with attention mechanism have shown good performance for answer selection in recent years. Most previous attention mechanisms focus on generating the attentive weights after obtaining all the hidden states, while the contextual information from the other sentence is not well studied during the internal hidden state generation. In this paper, we propose a position gated RNN (PG-RNN) model, which merges the positional contextual information of the question words for the inner hidden state generation. Specifically, we first design a positional interaction monitor to detect and measure the positional influence of question word within answer sentence. Then we present a positional gating mechanism and embed it into RNN to automatically absorb the positional contextual information for the hidden state update. Experiments on two benchmark datasets, namely TREC-QA and WikiQA, show the great advantages of our proposed model. In particular, we achieve the new state-of-the-art performance on TREC-QA and WikiQA.
更多查看译文
关键词
Position, Gate, Attention, Recurrent neural network
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络