This paper describes an approach for investigating the representation of events and their distribution in a corpus. We collect and analyze statistics about subject-verb-object triplets and their content, which helps us compare corpora belonging to the same domain but to different genre/text type. We argue that event structure is strongly related to the genre of the corpus, and propose statistical properties that are able to capture these genre differences. The results obtained can be used for the improvement of Information Extraction.

We present work on estimating the relevance of the results of an Event Extraction system to the end-user’s needs. Our aim is to develop user-oriented measures of utility of the extracted events, i.e., how useful is the factual information found in the document for the end user. We introduce discourse and lexical features, and build classifiers that learn from the users’ ratings of the relevance of the extraction results. Traditional criteria for evaluating the performance of Information Extraction (IE) focus on the correctness of the extracted information, e.g., in terms of recall, precision, F-measure, etc. We rather focus on subjective criteria for evaluating the quality of the extracted information: utility of results to the end-user. To measure utility, we use methods from text mining and linguistic analysis to identify features that are good predictors of the relevance of an event or a document. We report on experiments in two real-world event extraction domains: corporate activities reported in business news, and health threats in news about infectious epidemics.

We present on-going work on estimating the relevance of the results of an Information Extraction (IE) system. Our aim is to build a user-oriented measure of utility of the extracted factual information. We describe experiments using discourse-level features, with classifiers that learn from users’ ratings of relevance of the results. Traditional criteria for evaluating the performance of IE focus on correctness of the extracted information, e.g., in terms of recall, precision and F-measure. We introduce subjective criteria for evaluating the quality of the extracted information: utility of results to the end-user. To measure utility, we use methods from text mining and linguistic analysis to identify features that are good predictors of the relevance of an event or a document to a user. We report on experiments in two real-world news domains: business activity and epidemics of infectious disease.

This paper presents ongoing work on application of Information Extraction (IE) technology to domain of Public Health, in a real-world scenario. A central issue in IE is the quality of the results. We present two novel points. First, we distinguish the criteria for quality: the objective criteria that measure correctness of the system's analysis in traditional terms (F-measure, recall and precision), and, on the other hand, subjective criteria that measure the utility of the results to the end-user. Second, to obtain measures of utility, we build an environment that allows users to interact with the system by rating the analyzed content. We then build and compare several classifiers that learn from the user's responses to predict the relevance scores for new events. We conduct experiments with learning to predict relevance, and discuss the results and their implications for text mining in the domain of Public Health.

In epidemic surveillance, monitoring numerous languages is an important issue. In this paper we present a system designed to work on French, Spanish and English. The originality of our system is that we use only a few resources to perform our information extraction tasks. Instead of using ontologies, we use structure patterns of newspapers articles. The results on these three languages are encouraging at the preliminary stage and we will present a few examples of interesting experiments in other languages.

This presentation gives an overview of an effort to construct OSINT (Open-Source Intelligence) tools for Frontex, the European Agency for the Management of Operational Cooperation at the External Borders of the Member States of the European Union, to facilitate automating the process of extracting structured knowledge from on-line news articles on border-security related events at the EU borders and in related third countries. A particular focus is on incidents and developments which are of relevance in the context of illegal migration. This includes: (a) illegal migration incidents (e.g., illegal border crossing attempts), (b) related cross-border crime (e.g., human/arms/drug trafficking), (c) related crisis events (e.g., terrorist attacks, outbreaks of infectious disease).

This paper presents an endeavor aiming at construction of a real-time event extraction system for border security-related intelligence gathering from online news. First, the background and motivation behind the presented work is given. Next, the paper describes the event extraction processing chain, the specifics of the domain, i.e., illegal migration and related cross-border crime, and event moderation and visualisation aspects of the system.

This work demonstrates the ProMED-PLUS Epidemiological Fact Base. The facts are automatically extracted from plain-text reports about outbreaks of infectious epidemics around the world. The system collects new reports, extracts new facts, and updates the database, in real time. The extracted database is available on-line through a Web server.

This paper discusses/presents problems of template structure for Information Extraction. We investigate these problems in the context of two new Information Extraction scenarios which are linguistically and structurally more challenging than the traditional MUC scenarios. By a scenario we mean a predefined set of facts to be extracted from text. Traditional views on event structure and template design are not adequate for the more complex scenarios. We identify two structural factors that contribute to the complexity of a scenario: first, the scattering of events in text, and second, inclusion relationship between events. These factors cause difficulty in representing the facts in an unambiguous way. Traditional views on event structure and template design are not adequate for the more complex scenarios. We propose that these kinds of event relationships can be better described with a modular, hierarchical model.

This paper presents new Information Extraction scenarios which are linguistically and structurally more challenging than the traditional MUC scenarios. Traditional views on event structure and template design are not adequate for the more complex scenarios.The focus of this paper is to show the complexity of the scenarios, and propose a way to recover the structure of the event. First we identify two structural factors that contribute to the complexity of scenarios: the scattering of events in text, and inclusion relationships between events. These factors cause difficulty in representing the facts in an unambiguous way. Then we propose a modular, hierarchical representation where the information is split in atomic units represented by templates, and where the inclusion relationships between the units are indicated by links. Lastly, we discuss how we may recover this representation from text, with the help of linguistic cues linking the events.

We describe a system for creating and automatically updating a data base of information on infectious disease outbreaks. A web crawler is used to retrieve current news stories; potentially relevant stories are fed to an information extraction engine, whose output is used to update the data base. A web-based browser allows users to examine the data base and select records based on date, disease, location, and other fields. The browser can also display the source text from which the data base information is derived. The data base provides an alternative to conventional search engines for retrieving documents in this domain; the advantages and disadvantages of this approach are discussed.

Information Extraction (IE) systems are commonly based on pattern matching. Adapting an IE system to a new scenario entails the construction of a new pattern base-a time-consuming and expensive process. We have implemented a system for finding patterns automatically from un-annotated text. Starting with a small initial set of seed patterns proposed by the user, the system applies an incremental discovery procedure to identify new patterns. We present experiments with evaluations which show that the resulting patterns exhibit high precision and recall.

In developing an Information Extraction (IE) system for a new class of events or relations, one of the major tasks is identifying the many ways in which these events or relations may be expressed in text. This has generally involved the manual analysis and, in some cases, the annotation of large quantities of text involving these events. This paper presents an alternative approach, based on an automatic discovery procedure, EXDISCO, which identifies a set of relevant documents and a set of event patterns from un-annotaled text, starting from a small set of "seed patterns." We evaluate EXDISCO by comparing the performance of discovered patterns against that of manually constructed systems on actual extraction tasks.