AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
This paper proposes Biomedical Event Extraction as Sequence Labeling, a new end-to-end biomedical event extraction system which is both efficient and accurate

Biomedical Event Extraction as Sequence Labeling

EMNLP 2020, pp.5357-5367, (2020)

Cited by: 0|Views125
Full Text
Bibtex
Weibo

Abstract

We introduce Biomedical Event Extraction as Sequence Labeling (BeeSL), a joint end-to-end neural information extraction model. BeeSL recasts the task as sequence labeling, taking advantage of a multi-label aware encoding strategy and jointly modeling the intermediate tasks via multi-task learning. BeeSL is fast, accurate, end-to-end, and ...More

Code:

Data:

0
Introduction
  • Biomedical event extraction provides invaluable means for assisting domain experts in the curation of knowledge bases and biomolecular pathways (Ananiadou et al, 2010).
  • Events are typically highly complex and nested structures, which require deep contextual knowledge to resolve.
  • This is the case for biomedical NLP (Kim et al, 2011), where biomolecular events can be nested (Miwa et al, 2014) and long-distance arguments are frequent (Li et al, 2019).
  • There is a +REGULATION event triggered by the
Highlights
  • Biomedical event extraction provides invaluable means for assisting domain experts in the curation of knowledge bases and biomolecular pathways (Ananiadou et al, 2010)
  • We propose a new approach for biomedical event extraction by casting it as a sequence labeling task (BEESL)
  • In terms of sentences per minute, Biomedical Event Extraction as Sequence Labeling (BEESL) processes ∼500 sents/min compared to 255 sents/min and 101 sents/min in TEES single (3.42% lower F1) and ensemble (2.12% lower F1), respectively
  • This paper proposes BEESL, a new end-to-end biomedical event extraction system which is both efficient and accurate
  • BEESL is broadly applicable to event extraction and other tasks that can be recast as sequence labeling
  • Our analysis shows that BEESL works very well across event types
Methods
  • P R F1

    Riedel et al (2011) Miwa et al (2012) Venugopal et al (2014) Majumder et al (2016)

    FAUST – Model combination EventMine – SVM pipeline (+coref) BioMLN – SVM pipeline & MLN Stacked generalization

    Bjorne and Salakoski (2018) Bjorne and Salakoski (2018) Bjorne and Salakoski (2018)* Li et al (2019) Li et al (2019) Li et al (2019)

    TEES – CNN pipeline TEES – CNN pipeline (5x ensemble) TEES – CNN pipeline BiLSTM pipeline Tree-LSTM pipeline KB-driven Tree-LSTM pipeline

    Multi-task neural sequence labeling

    Multi-task d , r, h d, r , h d, h , r d, r, h Multi-label BEESLST

    BEESLMT outperforming the other MTL options, in recall.
  • Multi-task d , r, h d, r , h d, h , r d, r, h Multi-label BEESLST.
  • BEESLMT outperforming the other MTL options, in recall.
  • These results show that a multi-task setup with separate tasks for mention detection and head labeling, respectively, is the most useful.
  • Option 1, i.e., d , r, h defaults to the multi-task option for BEESL (Figure 3) used in the following experiments.
Results
  • The authors evaluate the MTL and multi-label decoding strategies on the development set to determine the best setup (Sections 5.1, 5.2).
  • As introduced in Section 3.2, this reduces to predicting the highest scoring label only – in a reduced label space induced by the multi-label aware decoder.
  • The authors found only part of the improvement is due to the threshold τ in both multi-task and single-task settings (+0.50% and +0.47%, respectively) (Table 7)
Conclusion
  • To gain insights about BEESL, the authors shed more light on several aspects. Firstly, the authors analyze how much BEESL gains from multi-task learning, compared to using a powerful contextualized BERT encoder alone in a single-task learning setup and a formulation with two independent classifiers (Section 6.1).
  • As opposed to running one single model which models d and r, h jointly in a multi-task setup, the authors compare to single-task (ST) and an experiment in which the authors formulate two classifiers which predict the two labels from the best MTL setup separately
  • This allows them to gauge the effectiveness of the multi-task learning approach compared to local classifiers which use strong BERT-based encoding, and compared to predicting an atomic label in ST.This paper proposes BEESL, a new end-to-end biomedical event extraction system which is both efficient and accurate.
  • The authors release the code freely, to foster research on using BEESL for other NLP tasks as well, e.g., enhanced dependency parsing, fine-grained named entity recognition, and semantic parsing
Summary
  • Introduction:

    Biomedical event extraction provides invaluable means for assisting domain experts in the curation of knowledge bases and biomolecular pathways (Ananiadou et al, 2010).
  • Events are typically highly complex and nested structures, which require deep contextual knowledge to resolve.
  • This is the case for biomedical NLP (Kim et al, 2011), where biomolecular events can be nested (Miwa et al, 2014) and long-distance arguments are frequent (Li et al, 2019).
  • There is a +REGULATION event triggered by the
  • Objectives:

    The authors aim to learn a function f : X → Y that assigns each token xi a structured label yi, i.e.,.
  • The main error the authors found may benefit from syntactic information, which the authors aim to integrate in a multitask setup in future work
  • Methods:

    P R F1

    Riedel et al (2011) Miwa et al (2012) Venugopal et al (2014) Majumder et al (2016)

    FAUST – Model combination EventMine – SVM pipeline (+coref) BioMLN – SVM pipeline & MLN Stacked generalization

    Bjorne and Salakoski (2018) Bjorne and Salakoski (2018) Bjorne and Salakoski (2018)* Li et al (2019) Li et al (2019) Li et al (2019)

    TEES – CNN pipeline TEES – CNN pipeline (5x ensemble) TEES – CNN pipeline BiLSTM pipeline Tree-LSTM pipeline KB-driven Tree-LSTM pipeline

    Multi-task neural sequence labeling

    Multi-task d , r, h d, r , h d, h , r d, r, h Multi-label BEESLST

    BEESLMT outperforming the other MTL options, in recall.
  • Multi-task d , r, h d, r , h d, h , r d, r, h Multi-label BEESLST.
  • BEESLMT outperforming the other MTL options, in recall.
  • These results show that a multi-task setup with separate tasks for mention detection and head labeling, respectively, is the most useful.
  • Option 1, i.e., d , r, h defaults to the multi-task option for BEESL (Figure 3) used in the following experiments.
  • Results:

    The authors evaluate the MTL and multi-label decoding strategies on the development set to determine the best setup (Sections 5.1, 5.2).
  • As introduced in Section 3.2, this reduces to predicting the highest scoring label only – in a reduced label space induced by the multi-label aware decoder.
  • The authors found only part of the improvement is due to the threshold τ in both multi-task and single-task settings (+0.50% and +0.47%, respectively) (Table 7)
  • Conclusion:

    To gain insights about BEESL, the authors shed more light on several aspects. Firstly, the authors analyze how much BEESL gains from multi-task learning, compared to using a powerful contextualized BERT encoder alone in a single-task learning setup and a formulation with two independent classifiers (Section 6.1).
  • As opposed to running one single model which models d and r, h jointly in a multi-task setup, the authors compare to single-task (ST) and an experiment in which the authors formulate two classifiers which predict the two labels from the best MTL setup separately
  • This allows them to gauge the effectiveness of the multi-task learning approach compared to local classifiers which use strong BERT-based encoding, and compared to predicting an atomic label in ST.This paper proposes BEESL, a new end-to-end biomedical event extraction system which is both efficient and accurate.
  • The authors release the code freely, to foster research on using BEESL for other NLP tasks as well, e.g., enhanced dependency parsing, fine-grained named entity recognition, and semantic parsing
Tables
  • Table1: Statistics of the Genia 2011 event dataset
  • Table2: Performance comparison on the test set of BioNLP Genia 2011. *indicates that the system was trained on training plus part of development data. BEESL uses the official training portion only. Top: traditional ML systems; Middle: state-of-the-art neural systems; Bottom: proposed multi-task sequence labeling system
  • Table3: top) summarizes the main results for the MTL experiments. They confirm our hypothesis that d , r, h (option 1) is the most viable representation; it leads to the highest F1 score, largely. Performance of diverse settings for BEESL (multi-task and multi-label) on the development set
  • Table4: Per-event performance of BEESL and KBTL (KB-driven TreeLSTM) (<a class="ref-link" id="cLi_et+al_2019_a" href="#rLi_et+al_2019_a">Li et al, 2019</a>) on the test set
  • Table5: Speed comparison to TEES (<a class="ref-link" id="cBjorne_2018_a" href="#rBjorne_2018_a">Bjorne and Salakoski, 2018</a>) single and ensemble models at inference time. Results are sents/min, averaged over 5 runs
  • Table6: Ablation study on BEESL when removing the multi-task capability (i.e., replacing MTL with independent classifiers) and the multi-label handling
  • Table7: Ablation study on the threshold τ of the multilabel decoder (“with best-only predicion”: τ = 1.0)
  • Table8: Performance of BEESL with no gold entities
  • Table9: Error analysis on a random sample of 30 documents from the development set
  • Table10: Formal definition of events. P: PROTEIN, E: any event type, +: 1 or more arguments
  • Table11: Hyper-parameter values and search space
  • Table12: Detailed per-event performance of BEESL and KBTL (KB-driven TreeLSTM) on the test set
Download tables as Excel
Related work
Funding
  • This research was supported by Fondazione the Microsoft Research – University of Trento Centre for Computational and Systems Biology, Italy, an Amazon Research Award, Independent Research Fund Denmark (Sapere Aude grant 9063-00077B), and NVIDIA corporation for sponsoring Titan GPUs
Study subjects and analysis
documents: 30
6.4 What are the sources of errors?. We randomly sampled 30 documents (comprising 168 gold events) from the development set for a manual scrutiny for sources of errors. We classified errors into two broad categories, namely trigger and argument errors

documents: 30
Performance of BEESL with no gold entities. Error analysis on a random sample of 30 documents from the development set. Formal definition of events. P: PROTEIN, E: any event type, +: 1 or more arguments

Reference
  • Zeljko Agicand Natalie Schluter. 2017. How (not) to train a dependency parser: The curious case of jackknifing part-of-speech taggers. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 679–684, Vancouver, Canada. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Christoph Alt, Marc Hubner, and Leonhard Hennig. 2019. Improving relation extraction by pre-trained language representations. In Proceedings of AKBC 2019.
    Google ScholarLocate open access versionFindings
  • Sophia Ananiadou, Sampo Pyysalo, Jun’ichi Tsujii, and Douglas Kell. 2010. Event extraction for systems biology by text mining the literature. Trends in biotechnology, 28(7):381–390.
    Google ScholarLocate open access versionFindings
  • Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018a. Adversarial training for multi-context joint entity and relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2830–2836, Brussels, Belgium. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018b. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications, 114:34– 45.
    Google ScholarFindings
  • Jari Bjorne and Tapio Salakoski. 2011. Generalizing biomedical event extraction. In Proceedings of BioNLP Shared Task 2011 Workshop, pages 183– 191, Portland, Oregon, USA. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Jari Bjorne and Tapio Salakoski. 2018. Biomedical event extraction using convolutional neural networks and dependency parsing. In Proceedings of the BioNLP 2018 workshop, pages 98–108, Melbourne, Australia. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • K Bretonnel Cohen, Helen L Johnson, Karin Verspoor, Christophe Roeder, and Lawrence E Hunter. 2010.
    Google ScholarFindings
  • The structural and content aspects of abstracts versus bodies of full text journal articles are different. BMC bioinformatics, 11(1):492.
    Google ScholarLocate open access versionFindings
  • Dai Dai, Xinyan Xiao, Yajuan Lyu, Shan Dou, Qiaoqiao She, and Haifeng Wang. 2019. Joint extraction of entities and overlapping relations using position-attentive sequence labeling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6300–6308.
    Google ScholarLocate open access versionFindings
  • Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1– 6, Melbourne, Australia. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Carlos Gomez-Rodrıguez and David Vilares. 2018. Constituent parsing as sequence labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1314– 1324, Brussels, Belgium. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Rob van der Goot, Ahmet Ustun, Alan Ramponi, and Barbara Plank. 2020. Massive choice, ample tasks (MaChAmp): A toolkit for multi-task learning in NLP. arXiv preprint arXiv:2005.14672.
    Findings
  • Jin-Dong Kim, Yue Wang, Toshihisa Takagi, and Akinori Yonezawa. 2011. Overview of Genia event task in BioNLP shared task 2011. In Proceedings of BioNLP Shared Task 2011 Workshop, pages 7–15, Portland, Oregon, USA. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pages 2779–2795, Hong Kong, China. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019. BioBERT: A pretrained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240.
    Google ScholarLocate open access versionFindings
  • Diya Li, Lifu Huang, Heng Ji, and Jiawei Han. 2019. Biomedical event extraction based on knowledgedriven tree-LSTM. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1421–1430, Minneapolis, Minnesota. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7999–8009, Online. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Amit Majumder, Asif Ekbal, and Sudip Kumar Naskar. 2016. Biomolecular event extraction using a stacked generalization based classifier. In Proceedings of the 13th International Conference on Natural Language Processing, pages 55–64, Varanasi, India. NLP Association of India.
    Google ScholarLocate open access versionFindings
  • David McClosky, Mihai Surdeanu, and Christopher Manning. 2011. Event extraction as dependency parsing. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1626–1635, Portland, Oregon, USA. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Makoto Miwa, Paul Thompson, and Sophia Ananiadou. 2012. Boosting automatic event extraction from the literature using domain adaptation and coreference resolution. Bioinformatics, 28(13):1759–1765.
    Google ScholarLocate open access versionFindings
  • Makoto Miwa, Paul Thompson, Ioannis Korkontzelos, and Sophia Ananiadou. 2014. Comparable study of event extraction in newswire and biomedical domains. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2270–2279, Dublin, Ireland. Dublin City University and Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Mark Neumann, Daniel King, Iz Beltagy, and Waleed Ammar. 2019. ScispaCy: Fast and robust models for biomedical natural language processing. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 319–327, Florence, Italy. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • 2017, pages 126–135, Vancouver, Canada,. Association for Computational Linguistics.
    Google ScholarFindings
  • Sebastian Riedel and Andrew McCallum. 2011. Robust biomedical event extraction with dual decomposition and minimal domain adaptation. In Proceedings of BioNLP Shared Task 2011 Workshop, pages 46–50, Portland, Oregon, USA. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Sebastian Riedel, David McClosky, Mihai Surdeanu, Andrew McCallum, and Christopher D. Manning. 2011. Model combination for event extraction in BioNLP 2011. In Proceedings of BioNLP Shared Task 2011 Workshop, pages 51–55, Portland, Oregon, USA. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5149–5152. IEEE.
    Google ScholarLocate open access versionFindings
  • Drahomıra Spoustovaand Miroslav Spousta. 2010. Dependency parsing as a sequence labeling task. The Prague Bulletin of Mathematical Linguistics, 94(1):7–14.
    Google ScholarLocate open access versionFindings
  • Michalina Strzyz, David Vilares, and Carlos GomezRodrıguez. 2019. Viable dependency parsing as sequence labeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 717–723, Minneapolis, Minnesota. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Deepak Venugopal, Chen Chen, Vibhav Gogate, and Vincent Ng. 2014. Relieving the computational bottleneck: Joint inference for event extraction with high-dimensional features. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 831–843, Doha, Qatar. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • David Vilares, Mostafa Abdou, and Anders Søgaard. 2019.
    Google ScholarFindings
  • Better, faster, stronger sequence tagging constituent parsers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3372–3383, Minneapolis, Minnesota. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Andreas Vlachos and Mark Craven. 2012. Biomedical event extraction from abstracts and full papers using search-based structured prediction. BMC Bioinformatics, 13(11):1759–1765.
    Google ScholarLocate open access versionFindings
  • Sudha Rao, Daniel Marcu, Kevin Knight, and Hal Daume III. 2017. Biomedical event extraction using abstract meaning representation. In BioNLP
    Google ScholarLocate open access versionFindings
  • Events on the Genia 2011 benchmark follow the formal specification detailed in Table 10. The full data can be downloaded from the official portal.6
    Google ScholarFindings
Author
Alan Ramponi
Alan Ramponi
Rob van der Goot
Rob van der Goot
Rosario Lombardo
Rosario Lombardo
Your rating :
0

 

Tags
Comments
小科