article Share on LibraryThing: an interview with Tim Spalding Author: Anna Ritchie View Profile Authors Info & Claims XRDS: Crossroads, The ACM Magazine for StudentsVolume 15Issue 4June 2009 Article No.: 2pp 3–6https://doi.org/10.1145/1558897.1558899Published:01 June 2009Publication History 0citation1,200DownloadsMetricsTotal Citations0Total Downloads1,200Last 12 Months7Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This thesis investigates taking words from around citations to scientific papers in order to create an enhanced document representation for improved information retrieval. This method parallels how anchor text is commonly used in Web retrieval. In previous work, words from citing documents have been used as an alternative representation of the cited document but no previous experiment has combined them with a full-text document representation and measured effectiveness in a large scale evaluation. The contributions of this thesis are twofold: firstly, we present a novel document representation, along with experiments to measure its effect on retrieval effectiveness, and, secondly, we document the construction of a new, realistic test collection of scientific research papers, with references (in the bibliography) and their associated citations (in the running text of the paper) automatically annotated. Our experiments show that the citation-enhanced document representation increases retrieval effectiveness across a range of standard retrieval models and evaluation measures. In Chapter 2, we give the background to our work, discussing the various areas from which we draw together ideas: information retrieval, particularly link structure analysis and anchor text indexing, and bibliometrics, in particular citation analysis. We show that there is a close relatedness of ideas between these areas but that these ideas have not been fully explored experimentally. Chapter 3 discusses the test collection paradigm for evaluation of information retrieval systems and describes how and why we built our test collection. In Chapter 4, we introduce the ACL Anthology, the archive of computational linguistics papers that our test collection is centred around. The archive contains the most prominent publications since the beginning of the field in the early 1960s, consisting of one journal plus conferences and workshops, resulting in over 10,000 papers. Chapter 5 describes how the PDF papers are prepared for our experiments, including identification of references and citations in the papers, once converted to plain text, and extraction of citation information to an XML database. Chapter 6 presents our experiments: we show that adding citation terms to the full-text of the papers improves retrieval effectiveness by up to 7.4%, that weighting citation terms higher relative to paper terms increases the improvement and that varying the context from which citation terms are taken has a significant effect on retrieval effectiveness. Our main hypothesis that citation terms enhance a full-text representation of scientific papers is thus proven. There are some limitations to these experiments. The relevance judgements in our test collection are incomplete but we have experimentally verified that the test collection is, nevertheless, a useful evaluation tool. Using the Lemur toolkit constrained the method that we used to weight citation terms; we would like to experiment with a more realistic implementation of term weighting. Our experiments with different citation contexts did not conclude an optimal citation context; we would like to extend the scope of our investigation. Now that our test collection exists, we can address these issues in our experiments and leave the door open for more extensive experimentation.

In previous work, we have shown that using terms from around citations in citing papers to index the cited paper, in addition to the cited paper's own terms, can improve retrieval effectiveness. Now, we investigate how to select text from around the citations in order to extract good index terms. We compare the retrieval effectiveness that results from a range of contexts around the citations, including no context, the entire citing paper, some fixed windows and several variations with linguistic motivations. We conclude with an analysis of the benefits of more complex, linguistically motivated methods for extracting citation index terms, over using a fixed window of terms. We speculate that there might be some advantage to using computational linguistic techniques for this task.

We present the results of experiments using terms from citations for scientific literature search. To index a given document, we use terms used by citing documents to describe that document, in combination with terms from the document itself. We find that the combination of terms gives better retrieval performance than standard indexing of the document terms alone and present a brief analysis of our results. This paper marks the first experimental results from a new test collection of scientific papers, created by us in order to study citation-based methods for IR.

Sutton Hoo: a seventh-century princely burial ground and its context. By Martin Carver. 295mm. Pp 536, 16 col, 124 b&w ills. Society of Antiquaries Research Report 69. London: The British Museum Press, 2005. ISBN 978-0-714123-22-6. £95 (hdbk). - Volume 87

We investigate the effect of different sources of relevant documents in the creation of a test collection in the scientific domain. Based on the Cranfield 2 design, paper authors are asked to judge their cited papers for relevance in the first stage. In a second stage, documents outside the reference list are judged. In this paper, we use the test collection with standard IR engines to compare the information contained in the judgements of the first vs second stage. Using different correlation studies, we found that the judgements of the cited papers do not predict those from the non-cited papers, which means that the combination of sources results in a higher quality collection.

We present an approach to building a test collection of research papers. The approach is based on the Cranfield 2 tests but uses as its vehicle a current conference; research questions and relevance judgements of all cited papers are elicited from conference authors. The resultant test collection is different from TREC's in that it comprises scientific articles rather than newspaper text and, thus, allows for IR experiments that include citation information. The test collection currently consists of 170 queries with relevance judgements; the document collection is the ACL Anthology. We describe properties of our queries and relevance judgements, and demonstrate the use of the test collection in an experimental setup. One potentially problematic property of our collection is that queries have a low number of relevant documents; we discuss ways of alleviating this.

We consider the question of how information from the textual context of citations in scientific papers could improve indexing of the cited papers. We first present examples which show that the context should in principle provide better and new index terms. We then discuss linguistic phenomena around citations and which type of processing would improve the automatic determination of the right context. We present a case study, studying the effect of combining the existing index terms of a paper with additional terms from papers citing that paper in our corpus. Finally, we discuss the need for experimentation for the practical validation of our claim.

An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.

Scotland in Ancient Europe: the Neolithic and Early Bronze Age of Scotland in their European Context. Edited by Ian A G Shepherd and Gordon J Barclay. 280mm. Pp xxi + 321, ills. Edinburgh: Society of Antiquaries of Scotland, 2004. ISBN 0903903318. £55 (hdbk). - Volume 86

Various applications could potentially benefit from the integration of deep and shallow processing techniques. A universal representation, compatible between deep and shallow parsers, would enable such integration, allowing the advantages of both to be combined. This paper describes efforts to make RMRS such a representation. This work was done as part of DeepThought, funded under the 5th Framework Program of the European Commission (contract reference IST-2001-37836).

Neolithic Settlement in Ireland and Western Britain. Edited by Ian Armit, Eileen Murphy, Eiméar Nelis and Derek Simpson. 310mm. Pp viii + 222, 116 b&w figs, 13 tables. Oxford: Oxbow Books, 2003. ISBN 1842170910. £35 (hdbk). - Volume 85

The Roundhouses, Brochs and Wheelhouses of Atlantic Scotland c. 700 BC – AD 500. Architecture and Material Culture. Part I: The Orkney and Shetland Isles. By Euan W Mackie. Pp xviii plus 388, ills. Oxford: BAR British Ser 342, 2002. ISBN 1841714593. £45. - Volume 84

Picts, Gaels and Scots: Early Historic Scotland. By Sally M. Foster. 248mm. Pp. 128, 88 ills., 12 col. pls. London: Batsford/Historic Scotland, 1996. ISBN 0-7134-7485-8 (h/b) £25.00. ISBN 0-7134-7486-6 (p/b) £15.99. - Stirling Castle. By Richard Fawcett. 248mm. Pp. 128, 100 ills., 15 col. pls. London: Batsford/Historic Scotland, 1995. ISBN 0-7134-7622-2 (h/b) £25.00. ISBN 0-7134-7623-0 (p/b) £15.99. - Volume 78

Exploring England's Heritage: Devon and Cornwall. By Andrew Saunders. 245 × 190mm. Pp. x + 117, maps, figs., pls. London: Her Majesty's Stationery Office in association with English Heritage, 1991. ISBN 0-11-300025-1. £9.95 (p/b). - Exploring England's Heritage: London. By Elain Harwood and Andrew Saint. 245 × 190mm. Pp. vii + 267, maps, pls. London: Her Majesty's Stationery Office in association with English Heritage. 1991. ISBN 0-11-300032-4. £15.95 (p/b). - Volume 71

St Magnus Cathedral and Orkney's Twelfth-Century Renaissance. Edited by Barbara E. Crawford. 255 × 185mm. Pp. xviii + 283, 28 figs., 53 pls. incl. 11 col. Aberdeen: Aberdeen University Press, 1988. ISBN 0-08-036580-9. £25κ·00. - Volume 69 Issue 1

Isbister: a Chambered Tomb in Orkney. By John W. Hedges. 29.5 × 21 cm. Pp. xx + 313, 81 ills., 70 tables. Oxford: British Archaeological Reports (British Series, 115), 1983. ISBN 0-86054-213-0. £15.00. - Tomb of the Eagles: a Window on Stone Age Tribal Britain. By John W. Hedges. 22 × 15 cm. Pp. xii + 224, 46 figs.+ 25 pls., 8 tables. London: John Murray, 1984. ISBN 0-7195-4119-0. £12.50. - Volume 65 Issue 2