We describe our participation in tasks 2, 4 and 5 of the DUC 2004 evaluation. For each task, we present the system(s) used, focusing on novel and newly developed aspects. We also analyze the results of the human and automatic evaluations.

Abstract: this paper was supportedby the Defense Advanced Research ProjectsAgency under TIDES grant NUU01-00-1-8919.Any opinions, findings, or recommendations arethose of the authors and do not necessarily reflectthe views of the funding agency

Recently, there have been significant advances in several areas of language technology, including clustering, text categorization, and summarization. However, efforts to combine technology from these areas in a practical system for information access have been limited. In this paper, we present Columbia's Newsblaster system for online news summarization. Many of the tools developed at Columbia over the years are combined together to produce a system that crawls the web for news articles, clusters them on specific topics and produces multidocument summaries for each cluster.

We present an evaluation of domainindependent natural language tools for use in the identification of significant concepts in documents. Using qualitative evaluation, we compare three shallow processing methods for extracting index terms, i.e., terms that can be used to model the content of documents. We focus on two criteria: quality and coverage. In terms of quality alone, our results show that technical term (TT) extraction [Justeson and Katz 1995] receives the highest rating. However, in terms of a combined quality and coverage metric, the Head Sorting (HS) method, described in [Wacholder 1998], outperforms both other methods, keyword (KW) and TT.