Semistructured data is of increasing importance in many application domains, but one of its core use cases is representing documents. Consequently, effectively retrieving information from semistructured documents is an important problem that has seen work from both the information retrieval (IR) and databases (DB) communities. Comparing the large number of retrieval models and systems is a non-trivial task for which established benchmark initiatives such as TREC with their focus on unstructured documents are not appropriate. This chapter gives an overview of semistructured data in general and the INEX initiative for the evaluation of XML retrieval, focusing on the most prominent Adhoc Search Track.
Financial news carry information about economical figures and indicators. However, these texts are mostly unstructured and consequently hard to be processed in an automatic way. In this paper, we present a representation formalism that supports a linguistic composition for machine learning tasks. We show an innovative approach to structuring financial texts by extracting principal indicators. Considering announcements in the monetary policy domain, we distinguish between attributes and their values and argue that attributes are to be represented as an aggregated set of economic terms, keeping their values as corresponding conditional expressions. We close with a critical discussion and future perspectives.
In this paper, we overview the main research approaches developed in the area of Keyword Search over Relational Databases. In particular, we model the process for solving keyword queries in three phases: the management of the user’s input, the search algorithms, the results returned to the user. For each phase we analyze the main problems, the solutions adopted by the most important system developed by researchers and the open challenges. Finally, we introduce two open issues related to multi-source scenarios and database sources handling instance not fully accessible.
For bridging the gap between information retrieval (IR) and databases (DB), this article focuses on the logical view. We claim that IR should adopt three major concepts from DB, namely inference, vague predicates and expressive query languages. By regarding IR as uncertain inference, probabilistic versions of relational algebra and Datalog yield very powerful inference mechanisms for IR as well as allowing for more flexible systems. For dealing with various media and data types, vague predicates form a natural extension of text retrieval methods to attribute values, thus switching from propositional to predicate logic. A more expressive IR query language should support joins, be able to compute aggregated results, and allow for restructuring of the result objects.
The black box application evaluation methodology described in this tutorial is applicable to a broad range of operational information retrieval (IR) applications. Contrary to popular, traditional IR evaluation approaches that are limited to measure the IR system performance on a test collection, the black box evaluation methodology considers an IR application in its entirety: the underlying system, the corresponding document collection, and its configuration/application layer. A comprehensive set of quality criteria is used to estimate the user’s perception of the application. Scores are assigned as a weighted average of results from tests that evaluate individual aspects. The methodology was validated in a small evaluation campaign. An analysis of this campaign shows a correlation between the testers’ perception of the applications and the evaluation scores. Moreover, functional weaknesses of the tested IR applications can be identified and then systematically targeted.
This lecture is intended to serve as an introduction to Information Retrieval (IR) effectiveness metrics and their usage in IR experiments using test collections. Evaluation metrics are important because they are inexpensive tools for monitoring technological advances. This lecture covers a wide variety of IR metrics (except for those designed for XML retrieval, as there is a separature lecture dedicated to this topic) and discusses some methods for evaluating evaluation metrics. It also briefly covers computer-based statistical significance testing. The takeaways for IR experimenters are: (1) It is important to understand the properties of IR metrics and choose or design appropriate ones for the task at hand; (2) Computer-based statistical significance tests are simple and useful, although statistical significance does not necessarily imply practical significance, and statistical insignificance does not necessarily imply practical insignificance; and (3) Several methods exist for discussing which metrics are “good,” although none of them is perfect.
Semantic search over documents is about finding information that is not based just on the presence of words, but also on their meaning [1, 2]. This task is a modification of classical Information Retrieval (IR), but documents are retrieved on the basis of relevance to ontology concepts, as well as words. Nevertheless the basic assumption is quite similar - a document is characterized by the bag of tokens constituting its content, disregarding its structure. While the basic IR approach considers word stems as tokens, there has been considerable effort towards using word-senses or lexical concepts (see [3, 4]) for indexing and retrieval. In the case of semantic search, what is being indexed is typically a combination of words, ontological concepts conveying the meaning of some of these words (e.g. Cambridge is a location), and optionally relations between such concepts (e.g. Cambridge is in the UK) [1]. The latter enable somebody searching for documents about the UK to find also documents mentioning Cambridge.
How can the search process on Twitter be improved to better meet the various information needs of its users? As an answer to this question, we have developed the Twinder framework, a scalable search system for Twitter streams. Twinder contains algorithms to determine the relevance of tweets in relation to search requests, as well as components to detect (near-)duplicate content, to diversify search results, and to personalize the search result ranking. In this paper, we report on our current progress, including the system architecture and the different modules for solving specific problems. Finally, we empirically determine the effectiveness of Twinder’s components with experiments on representative datasets.
Cross-language plagiarism detection attempts to identify and extract automatically plagiarism among documents in different languages. Plagiarized fragments can be translated verbatim copies or may alter their structure to hide the copying, which is known as paraphrasing and is more difficult to detect. In order to improve the paraphrasing detection, we use a knowledge graph-based approach to obtain and compare context models of document fragments in different languages. Experimental results in German-English and Spanish-English cross-language plagiarism detection indicate that our knowledge graph-based approach offers a better performance compared to other state-of-the-art models.
This paper presents a selection of methods for searching in heterogeneous data collections where some amount of structure is available. We start with a general retrieval framework, based on generative probabilistic modeling, for ranking unstructured document representations. Then, we consider structure at two different levels: documents and queries. For documents, the internal structure is captured through the use of multiple document fields, and various approaches to setting field weights are discussed. For queries, the focus is on effectively utilizing additional input data that the user might provide along with the keyword query, such as target categories or example documents. We place a particular emphasis on methods that are robust with respect to the availability of structured data and are able to deal with inconsistent or incomplete information.
Visual Analytics (VA) [1] is an emerging multi-disciplinary area that takes into account both ad-hoc and classical Data Mining (DM) algorithms and Information Visualization IV (IV) techniques, combining the strengths of human and electronic data processing. Visualisation becomes the medium of a semi-automated analytical process, where human beings and machines cooperate using their respective distinct capabilities for the most effective results. Decisions on which direction analysis should take in order to accomplish a certain task are left to the user. Although IV techniques have been extensively explored [2], combining them with automated data analysis for specific application domains is still a challenging activity [3]. This chapter provides an introduction of the main concepts behind VA and presents some practical examples on how apply it to Information Retrieval (IR).
The field of information retrieval (IR) has experienced tremendous growth over the years. Researchers have however identified Human-Computer Interaction (HCI) aspects as important concerns in IR research. Incorporation of HCI techniques in IR can ensure that IR systems intended for human users are developed and evaluated in a way that is consistent with and reflects the needs of those users. The traditional methods of evaluating IR systems have for a long period been largely concerned with system-oriented measurements such as precision and recall, but not on the usability aspects of the IR system. There also are no well-established evaluation approaches for studying users and their interactions with IR systems. This chapter describes the role and place of HCI toward supporting and appropriating the evaluation of IR systems.
This note presents an overview of the literature related to multimedia information retrieval, as a tool in the context of inter-connected media. The goal is to propose and motivate a structure to present key focus and successful achievements in the domain. We go through the base foundations of multimedia information retrieval and investigate new challenges. Here, we particularly focus on providing large-scale accurate access to the data from both the user and the computation perspectives. We identify and discuss information representation and fusion as key building blocks of an efficient and accurate information access strategy.
The paper discusses briefly user-oriented evaluation in test collections with simulated users and real users, as well as operational systems evaluation. It concludes by a glimpse of issues beyond evaluation. The paper provides pointers to literature where much more thorough discussion of each topic may be found.
This text gives a broad overview of the domain of visual medical information retrieval and medical information analysis/search in general. The goal is to describe the specifics of medical information analysis and more specifically of medical visual information retrieval in this book of the PROMISE winter school. The text is meant to deliver an annotated bibliography of important papers and tendencies in the domain that can then guide the reader to find more detailed information on this quickly developing research domain. This text is by no means a systematic review in the field, so some citations might be subjective but should lead the reader to further publications. The given references will provide a solid starting point for exploring the domain of medical visual information retrieval.
Language and multimedia technology research often relies on large manually constructed datasets for training or evaluation of algorithms and systems. Constructing these datasets is often expensive with significant challenges in terms of recruitment of personnel to carry out the work. Crowdsourcing methods using scalable pools of workers available on-demand offers a flexible means of rapid low-cost construction of many of these datasets to support existing research requirements and potentially promote new research initiatives that would otherwise not be possible.
Since the '80s, Human-Computer Interaction (HCI) researchers have performed a lot of work to identify principles, techniques, and methodologies that can support design, evaluation and implementation of interactive systems that fulfill needs and expectations of their users. This chapter discusses concepts, such as usability and user experience, which are of great importance for the success of interactive systems, illustrating how Human-Centred Design is fundamental to create successful user interfaces. Principles proposed by the HCI community to support interface design are presented, analyzing the principles that have a major impact on IR interfaces.
Information visualization is defined as an interactive and graphic amplifying cognition. Moreover, the field of information retrieval is the original scope of information visualization. Nevertheless, many problems remain. The exploratory research information is presented as a task context conducive to the use of visualization. The research aims to identify the dynamic and interactive graphics that use a search visual interface (versus text), and the gain in terms of usability compared to strategic and tactical requirements of the task of exploratory search on the Web exploratory type. The theoretical, conceptual and methodological framework is presented.
