AI helps you reading Science
AI generates interpretation videos
AI extracts and analyses the key points of the paper to generate videos automatically
AI parses the academic lineage of this thesis
AI extracts a summary of this paper
The RikiNet consists of a dynamic paragraph dual-attention reader which learns the token-level, paragraphlevel and question representations, and a multilevel cascaded answer predictor which jointly predicts the long and short answers in a cascade manner
RikiNet: Reading Wikipedia Pages for Natural Question Answering
ACL, pp.6762-6771, (2020)
Reading long documents to answer open-domain questions remains challenging in natural language understanding. In this paper, we introduce a new model, called RikiNet, which reads Wikipedia pages for natural question answering. RikiNet contains a dynamic paragraph dual-attention reader and a multi-level cascaded answer predictor. The rea...More
- Machine reading comprehension (MRC) refers to the task of finding answers to given questions by reading and understanding some documents.
- It represents a challenging benchmark task in natural language understanding (NLU).
- There are two main challenges in NQ compared to the previous MRC datasets like SQuAD 2.0.
- NQ task requires the model to find an answer span to the question like previous MRC tasks and asks the model to find a paragraph that contains the information required to answer the question
- Machine reading comprehension (MRC) refers to the task of finding answers to given questions by reading and understanding some documents
- We focus on the Natural Questions task and propose a new Machine reading comprehension model called RikiNet tailored to its associated challenges, which Reads the Wikipedia pages for natural question answering
- We report the results of the precision (P), the recall (R), and the F1 score for the long-answer (LA) and short-answer (SA) tasks on both test set and dev set in Tab. 1
- The first two lines of Tab. 1 show the results of two multi-passage Machine reading comprehension baseline models presented in the original Natural Questions paper (Kwiatkowski et al, 2019)
- We propose the RikiNet, which reads the Wikipedia pages to answer the natural question
- The RikiNet consists of a dynamic paragraph dual-attention reader which learns the token-level, paragraphlevel and question representations, and a multilevel cascaded answer predictor which jointly predicts the long and short answers in a cascade manner
- The authors propose the RikiNet which Reads the Wikipedia pages for natural question answering.
- As shown in Fig. 1, RikiNet consists of two modules: (a) the dynamic paragraph dual-attention reader as described in §3.1, and (b) the multi-level cascaded answer predictor as described in §3.2.
- Dynamic Paragraph Dual-Attention (DPDA) reader aims to represent the document span d and the question q.
- It outputs the context-aware question representation, question-aware token-level document representation, and paragraph-level document representation, which will be all fed into the predictor to obtain the long and short answers.
- The authors present a comparison between previously published works on the NQ task and the RikiNet.
- The third to sixth lines show the results of the previous state-of-theart models
- These models all employ the BERTlarge model and perform better than that two baselines.
- The authors' single model of RikiNet-RoBERTa large which employs RoBERTa large model achieves better performance on both LA and SA, significantly outperforming BERTjoint + RoBERTa large.
- These results demonstrate the effectiveness of the RikiNet
- The authors propose the RikiNet, which reads the Wikipedia pages to answer the natural question.
- The RikiNet consists of a dynamic paragraph dual-attention reader which learns the token-level, paragraphlevel and question representations, and a multilevel cascaded answer predictor which jointly predicts the long and short answers in a cascade manner.
- On the Natural Questions dataset, the RikiNet is the first single model that outperforms the single human performance.
- The RikiNet ensemble achieves the new state-of-the-art results at 76.1 F1 on long-answer and 61.3 F1 on shortanswer tasks, which significantly outperforms all the other models on both criteria
- Table1: Performance comparisons on the dev set and the blind test set of the NQ dataset. We report the evaluation results of the precision (P), the recall (R), and the F1 score for both long-answer (LA) and short-answer (SA) tasks. We use background color to highlight the column of F1 results. † refers to the works that only provide the F1 results on the dev set in their paper. ‡ refers to our implementations where we only report the results on the dev set, due to the NQ leaderboard submission rules (each participant is only allowed to submit once per week)
- Table2: Ablations of DPDA reader on dev set of NQ dataset
- Table3: Ablations of multi-level cascaded predictor on dev set of NQ dataset
- Rajpurkar et al, 2018), a similar BERT method (Alberti et al, 2019b) still has a big gap with human performance on NQ dataset.
There are several recently proposed deep learning approaches for multi-passage reading comprehension. Chen et al (2017) propose DrQA which contains a document retriever and a document reader (DocReader). Clark and Gardner (2018) introduce Document-QA which utilizes TF-IDF for paragraph selection and uses a shared normalization training objective. De Cao et al (2019) employ graph convolutional networks (GCNs) for this task. Zhuang and Wang (2019) design a gated tokenlevel selection mechanism with a local convolution. In contrast, our RikiNet considers multi-level representations with a set of complementary attention mechanisms.
- This work is supported by National Natural Science Fund for Distinguished Young Scholar (Grant No 61625204) and partially supported by the Key Program of National Science Foundation of China (Grant No 61836006)
- Chris Alberti, Daniel Andor, Emily Pitler, Jacob Devlin, and Michael Collins. 2019a. Synthetic qa corpora generation with roundtrip consistency. arXiv preprint arXiv:1906.05416.
- Chris Alberti, Kenton Lee, and Michael Collins. 2019b. A bert baseline for the natural questions. arXiv preprint arXiv:1901.08634.
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
- Benjamin Borschinger, Jordan Boyd-Graber, Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Michelle Chen Huebscher, Wojciech Gajewski, Yannic Kilcher, Rodrigo Nogueira, and Lierni Sestorain Saralegu. 2019. Meta answering for machine reading. arXiv:1911.04156.
- Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer opendomain questions. In ACL.
- Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. In ACL.
- Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. 201Attention-overattention neural networks for reading comprehension. In ACL.
- Nicola De Cao, Wilker Aziz, and Ivan Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In NAACL-HLT.
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL.
- Michael Glass, Alfio Gliozzo, Rishav Chakravarti, Anthony Ferritto, Lin Pan, GP Bhargav, Dinesh Garg, and Avirup Sil. 2019. Span selection pretraining for question answering. arXiv preprint arXiv:1909.04120.
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR.
- Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415.
- Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. Neural computation.
- Ying Ju, Fubang Zhao, Shijie Chen, Bowen Zheng, Xuefeng Yang, and Yunfeng Liu. 2019. Technical report on conversational question answering. arXiv preprint arXiv:1909.10772.
- Diederik P Kingma and Jimmy Ba. 20Adam: A method for stochastic optimization. In ICLR.
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. TACL, 7:453– 466.
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
- Lin Pan, Rishav Chakravarti, Anthony Ferritto, Michael Glass, Alfio Gliozzo, Salim Roukos, Radu Florian, and Avirup Sil. 20Frustratingly easy natural question answering. arXiv preprint arXiv:1909.05286.
- Ankur P Parikh, Oscar Tackstrom, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In EMNLP.
- Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for squad. In ACL.
- Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In EMNLP.
- Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. Coqa: A conversational question answering challenge. TACL, 7:249–266.
- Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing.
- Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In ICLR.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS.
- Caiming Xiong, Victor Zhong, and Richard Socher. 2017. Dynamic coattention networks for question answering. In ICLR.
- Caiming Xiong, Victor Zhong, and Richard Socher. 2018. Dcn+: Mixed objective and deep residual coattention for question answering. In ICLR.
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237.
- Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, and Hai Zhao. 2019. Sg-net: Syntax-guided machine reading comprehension. arXiv preprint arXiv:1908.05147.
- Yimeng Zhuang and Huadong Wang. 2019. Tokenlevel dynamic self-attention network for multipassage reading comprehension. In ACL.