Events of geopolitical interest can unfold quickly and are frequently surrounded by uncertainty. This can largely be attributed to the emergence of information across various modalities and media in different languages, and is frequently compounded by the presence of conflicting claims. The crash of flight MH17 in Ukraine in 2014 serves as a good example. After the crash, a multitude of explanations arose, including Russia-affiliated aggression, Ukranian military aggression, an act of terror, and mechanical failure. The Streaming Multimedia Knowledge Base Population (SM-KBP) track at TAC 2019 continues to address the challenges of this scenario in three tasks. Task 1 is the extraction of relevant information from multi-modal and multi-lingual sources. Task 2 is the aggregation of document-level knowledge into a multi-document knowledge graph, including cross-document coreference resolution. Task 3 is to probe the resulting knowledge graph for information relevant to a provided statement of information need (SIN), entailing the separation of conflicting claims and the construction of internally consistent narratives (represented as subgraphs of the multi-document knowledge graph called hypotheses). The UTexas team participated in Task 3, hypothesis generation. We view hypothesis generation as a one-class clustering task (Bekkerman and Crammer, 2008) of detecting further coherent entities and statements surrounding a hypothesis seed. The UTexas system for TAC 2019 uses a hybrid system in which a neural model proposes statements for inclusion in the hypothesis cluster, and a rule-based filter that rejects statements based on either domain knowledge or domain-independent well-formedness criteria.
For fine-grained entity extraction, we propose a fine-grained entity typing model with a novel attention mechanism and a hybrid type classi-fier. We advance existing methods in two aspects: feature extraction and type prediction. To capture richer contextual information, we adopt contextualized word representations instead of fixed word embeddings used in previous work. In addition, we propose a two-step mention-aware attention mechanism to enable the model to focus on important words in mentions and contexts. We also develop a hybrid classification method beyond binary relevance to exploit type interdependency with latent type representation. Instead of independently predicting each type, we predict a low-dimensional vector that encodes latent type features and reconstruct the type vector from this latent representation.
The Text Analysis Conference (TAC) Drug-Drug Interaction Extraction from Drug Labels track is an evaluation of Natural Language Processing (NLP) techniques for identifying drugdrug interactions in Food & Drug Administration (FDA) Structured Product Labeling (SPL) documents. We participated in all four tasks corresponding to: (1) entity identification; (2) sentencelevel relation identification; (3) concept normalization; and (4) normalized relation identification. We developed a multi-task neural architecture based on BERT to jointly perform tasks (1), (2), and part of task (4). BERT is used to generate a contextualized representation for each word-piece token in a given sentence, which is shared between three prediction modules, the Boundary CRF (BCRF), the Relation classifier (RC), and the Pharmacokinetic Effect classifier (PKEC). Task (3) was performed using string matching on a MED-RT/UNII/SNOMED lexicon. The normalized interactions produced for Task (4) were entailed by the results of tasks (2) and (3) and the Pharmacokinetic effect codes identified by the PKEC. The results indicate that for tasks (1) and (2), a pipeline of special purpose systems out-perform joint learning. However, for task (4) – the focus of the track – the joint learning approach outperforms the pipeline.
Drug-Drug Interactions (DDI), which lead to adverse events, have been identified as the eighth leading cause of death in the United States (Goldstein et al., 2005). Structured Product Labeling (SPL) documents are a rich source of information for drug labels, but it is essential to convert them to discrete, coded information in order to enable automatic extraction of drug interactions. TAC 2019 DDI track defined multiple Natural Language Processing (NLP) tasks, namely concept extraction of Precipitant and SpecificInteraction (Task 1), relation extraction of interactions and their types (Task 2), concept normalization (Task 3) and generation of a global list of interactions per SPL document (Task 4). We participated in Tasks 1 and 2, where we used a combination of a novel tagging scheme, a fine-tuned transformer-based language model, and a syntactic-parse based knowledge-injected pattern matching technique. We submitted three systems for both Tasks 1 and 2. We achieved an F1-score of 65.38, 64.62 and 65.18 for Task 1, and an F1-score of 49.03, 48.33 and 48.39 for Task 2. Our system ranked first, with the highest F1-score for both Tasks 1 and 2, thus demonstrating an effective adaptation of our hybrid system on the DDI extraction tasks.
1.2. TAC EDL 2015-2017 and OntoNotes Compared to the YAGO data set, the TAC and OntoNotes data sets are much smaller. Each training set contains less than one hundred thousand sentences, but they are cleaner and contain no nested entities nor mentions of several entity types. A good reason to make use of the TAC corpus is that the five entity types in TAC EDL 2015-2017 (PER, ORG, GPE, LOC and FAC) match five of the first level entity types in AIDA, our target annotation system. OntoNotes uses another ontology and contains 18 entity types. In our previous experiments, however, we found that incorporating re-tagged OntoNotes data into the TAC EDL 2015 training data improves the system performance on the TAC EDL 2017 test set. Thus, we use these corpora as a seed for training a model on TAC ontology, and apply this model to generate synthetic annotations on sentences from YAGO.
HITS and the UKP Lab participated in the Entity Discovery and Linking Track at TAC KBP 2019. The main tasks were Named Entity Detection and Fine Grained Entity Typing, with a large inventory of entity types provided by DARPA AIDA (Active Interpretation of Disparate Alternatives). To address the task we apply a pipeline setup of named entity detection followed by entity typing, which aims to profit from the hierarchical relations present on the type inventory. We provide a description of our modular system and preliminary results on silver data.
This paper describes Studio Ousia’s participation to the EDL track of TAC KBP 2019— (ultra) fine-grained named entity recognition (NER). The proposed system first trains a YAGO-based ultra fine-grained NER model in a multi-label, multi-task fashion. The pre-trained model is then fine-tuned to adopt it to the AIDA taxonomy by adding a lightweight conversion layer. The experiments have shown that this transfer learning approach outperforms a simpler direct method which directly trains the YAGO-based NER model.
We designed and constructed a pipeline system for the Streaming Multimedia Knowledge Base Population (SM-KBP) 2019 evaluation. Our pipeline consists of a series of Information Extraction and Machine Translation components, supporting the population of knowledge graphs from a multilingual corpus (see Figure 1). Some of our key contributions include: a new dataset supporting event argument linking across multiple sentences; a novel algorithm for predicting such linkages, even when arguments are not co-referent with mentions in the same sentence as the event trigger; and a novel algorithm for supporting hierarchical typing of events, relations and entities.
The SRCB team participated in entity recognition task, relation identification task and normalization task in TAC Drug-Drug Interaction Extraction (DDI) 2019. The entity recognition system and relation identification system are based on BioBERT architecture with task-specific improvement. The normalization system includes candidate generation and further re-rank algorithms to find the right answer.
In this paper we describe our submission to the TAC KBP: Entity Linking and Discovery 2019 task. The task aims at extracting fine-grained mentions of 190 types from text where no training data is available for these types. Our submissions are based on a two pass approach: first detect nine basic NER types, followed by a second pass of refining these types into the destination fine-grained types.
In this report, we give a detailed description of Diffbot’s system for the TAC-KBP 2019 Fine-Grained Entity Typing track. Fine-grained entity typing is a task to extract mentions of pre-defined types and assign fine-grained types to those mentions. Instead of modeling the task as a multilabel classification problem, we tackle it using Entity Linking (EL) by first linking mentions to their referent entities in a Knowledge Base (KB) and then inferring the fine-grained types of mentions based on the linking results and type ontology.
This paper describes the Drug-Drug Interaction Extraction from Drug Labels Track, part of the 2019 Text Analysis Conference (TAC). Participants were provided with an annotated set of interactionsrelated sections of drug labels and challenged with: (1) extracting mentions of the precipitants and effects of drug-drug interactions at the sentence level; (2) identifying (typed) relations between interacting substances; (3) normalizing mentions and relations to several standard terminologies; and (4) determining the unique set of drug-drug interactions across all provided sections of a drug label. Four teams submitted at least one run, with 10 submissions in total.
This document describes SAMSON, Raytheon BBN and partners’ system used in the KB Construction and Hypothesis Generation tasks of the 2019 TAC SM-KBP competition. For KB Construction, SAMSON performs entity linking and disambiguation using vector embeddings of the KB. For Hypothesis Generation, SAMSON uses two different methods of relaxed querying. The first method constructs spaces of hierarchical clusters over various classes in the KB. We then translate both the KB and the query to cluster-level semantics and use a Prolog meta-interpreter to run the query. The second method uses local graph search to find KB nodes similar to those requested in the query.
Events and situations unfold quickly in our modern world, generating streams of Internet articles, photos, and videos. The ability to automatically sort through this wealth of information would allow us to identify which pieces of information are most important and credible, and how trends unfold over time. In this paper, we present the first piece of a system to sort through large amounts of political data from the web. Our system takes in raw multimodal input (e.g., text, images, and videos), and generates a knowledge graph connecting entities, events, and relations in meaningful ways. This work is part of the DARPA-funded Active Interpretation of Disparate Alternatives (AIDA) project, which aims to automatically build a knowledge base that can be queried to strategically generate hypotheses about different aspects of an event. We are participating in this project as a TA1 team, building the first step of the overall system. Our approach is outlined in Figure 1 and will be discussed in detail in the following sections. The first step of the pipeline is pre-processing, shown in
In the past several years TAC KBP Entity Discovery and Linking (EDL) track has only focused on five major coarse-grained entity types: person (PER), geo-political entity (GPE), location (LOC), organization (ORG) and facility (FAC). However, many real-world applications in scenarios such as disaster relief and technical support require us to significantly extend EDL capabilities to a wider variety of fine-grained entity types (e.g., technical terms, lawsuits, disease, crisis, vehicles, food, biomedical entities). In this overview paper we give an outline of the Ultra-Fine-Grained Name Tagging for Entity Types task (along with system participation) at the Knowledge Base Population (KBP) track at TAC 2019. We will also sketch out remaining challenges and future research directions.
Extraction and Hypothesis Generation System Manling Li, Ying Lin, Ananya Subburathinam, Spencer Whitehead, Xiaoman Pan, Di Lu, Qingyun Wang, Tongtao Zhang, Lifu Huang, Heng Ji 1 University of Illinois at Urbana-Champaign hengji@illinois.edu Alireza Zareian, Hassan Akbari, Brian Chen, Bo Wu, Emily Allaway, Shih-Fu Chang, Kathleen McKeown 2 Columbia University sc250@columbia.edu, kathy@cs.columbia.edu Yixiang Yao, Jennifer Chen, Eric Berquist, Kexuan Sun, Xujun Peng, Ryan Gabbard Marjorie Freedman, Pedro Szekely, T.K. Satish Kumar 3 Information Sciences Institute, University of Southern California mrf@isi.edu Arka Sadhu, Ram Nevatia University of Southern California nevatia@usc.edu Miguel Rodriguez5, Yifan Wang5, Yang Bai5, Ali Sadeghian4, Daisy Zhe Wang5 5 University of Florida daisyw@ufl.edu
