The rapidly increasing popularity of LLM-powered chatbots has led to them being used for a increasing number of different tasks by the general public. One of these tasks is searching for information instead of using a search engine. Previous work has shown that complex search tasks can be problematic for traditional search engines to solve, but little is known about the capability of LLMs on the same task. We compared four LLMs on their capability to answer a specific type of complex search task: known-item requests from casual leisure domains. We constructed a test collection by gathering known-item requests for books, games and movies from online forums along with verified answers by the original requester. We prompted four LLMs multiple times with the same prompt and analyzed the results with respect to accuracy and the degree to which answers were fabricated by the LLM. Our results show that LLMs are not particularly effective in fulfilling these complex casual leisure needs, but there are are big differences between LLMs and across domains.

Across all scientific fields, there is an increased focus on the impact of scientific research: what academic and societal benefits does it provide? This question has spurred the development of a variety of different approaches to impact assessment, each appropriate in different circumstances. In this paper, we study the academic impact of the CHIIR community through a comprehensive analysis of the work published in the 2016-2023 CHIIR conference series. We collect citation counts, citing documents, and altmetrics scores for all CHIIR publications to determine their academic impact across a variety of different attributes of the CHIIR publications. In addition, we analyze a subset of citation contexts in the papers that have cited CHIIR publications to analyze how they are being used and what that means for their potential impact. Finally, we attempt to predict which properties of CHIIR publications are most predictive of future impact.

We studied the collaboration patterns of CHIIR authors, and found that most papers are collaborative. A core of 33% of the CHIIR researchers are directly connected and frequently co-author, and several disconnected clusters also make frequent CHIIR contributions. We also studied citation impact of the CHIIR papers and show that in relation to research design type, theoretical and empirical papers tend to receive more citations than resource papers. With regards to sharing and re-use, papers that share at least one resource tend to have significantly higher citation impact—in particular when sharing data resources and design resources. Re-using resources does not significantly increase citation impact in itself.

In this paper, we present the results of an initial study of the research, sharing, and re-use practices at the CHIIR conference through a systematic analysis of all CHIIR papers published from 2016 to 2022. We find that CHIIR is a conference predominantly focused on empirical, multi-methods research that over the years has undergone a focusing in terms of the type of research methods that are being used. A modest number of papers re-use existing data and design resources, but infrastructure component re-use is much more rare. Only a fraction of CHIIR papers actually share their own resources, which suggests that there is much to gain in terms of reproducibility of research presented at CHIIR and could potentially be used to support changes in reviewing practices.

ACM Reference Format: Toine Bogers, Maria Gäde, Mark Hall, Marijn Koolen, Vivien Petras, and Paul Thomas. 2022. ThirdWorkshop on Building towards Information Interaction and Retrieval Resources Re-use (BIIRRR 2022). In Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22), March 14–18, 2022, Regensburg, Germany. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3498366.3505838

Digitisation of our cultural heritage has created vast digital archives, many of which are publicly accessible via the web. However, publicly accessible does not necessarily mean that the content of the collection is truly accessible to the wider public. The reason for this is that the standard interface for accessing the collections is the search box. This is very effective for experienced users who have the information seeking skills and domain knowledge to formulate appropriate queries, but for the less knowledgeable users the white search box represents a significant hurdle. This group of users require a generous interface that provides them with an overview over the collection and the ability to explore it, without having to explicitly enter a search keyword. In this chapter we discuss a range of existing approaches that have been used to provide less knowledgeable users with such interfaces and then present the Digital Museum Map, an algorithm and interface for automatically generating a virtual museum interface for an unstructured collection, that can then be explored by browsing through the virtual museum.

„Kulturen digitalen Gedächtnisses“ operieren Hintergrund eines Spannungsfelds, das eine Dichotomie Kanon Archiv zuspitzen Das digitale Archiv umfasst die digitalisierten Artefakte Kanon verschiedenen Gründen besonders bewahrenswerten Artefakte spezifischen Stellenwert Sie Bestandteil kulturellen digitale epistemische eine klassischen Bildungsinsti-tutionen, Wissen di-gitalisierte Archiv die Grundlage für historisch arbeitende Wissenschaften, Methoden Das Archiv digitalisiertes Gedächtnis einerseits altes Wissen und dient andererseits auch als Grundlage, von der aus, und mit der, neues Wissen produziert

Natural language descriptions of geographical locations are used frequently in daily life and there is a motivation to create systems that generate such descriptions automatically, for purposes such as documentation of where events have taken place, where a person is located, where photos were taken and where plants and animals are located. Typically location descriptions combine references to named geographical features with vague spatial relational terms, such as near, north of and at that relate locations to the features. Here we describe a system for generating location descriptions, that combines spatial templates, that model the applicability of different spatial relations relative to a reference location, with toponyms in the vicinity of the described location that are selected according to aspects of salience. The toponyms are retrieved from a gazetteer service based on OpenStreetMap for which we create a hierarchical feature classification scheme to facilitate selection of toponyms according to distinctiveness of their feature types and other aspects of salience. The advantages of the approach are demonstrated in a user study, relative to an existing state of the art system and to other baseline approaches that include manually created captions and the automated methods of two widely used photo captioning systems.

This perspective paper on resource re-use intends to draw the attention of the interactive information retrieval (IIR) community to the challenges of research documentation and archiving for future use. Resources are understood as encompassing research designs, research data and research infrastructures. It proposes eight principles for improving the re-use of resources in the IIR community and presents concrete steps on how to achieve them. A five-level system for data archiving and documentation envisions increasingly open and stable documentation and access infrastructures.

Museum websites have been designed to provide access for different types of users, such as museum staff, teachers and the general public. Therefore, understanding user needs and demographics is paramount to the provision of user-centred features, services and design. Various approaches exist for studying and grouping users, with a more recent emphasis on data-driven and automated methods. In this paper, we investigate user groups of a large national museum’s website using multivariate analysis and machine learning methods to cluster and categorise users based on an existing user survey. In particular, we apply the methods to the dominant group - general public - and show that sub-groups exist, although they share similarities with clusters for all users. We find that clusters provide better results for categorising users than the self-assigned groups from the survey, potentially helping museums develop new and improved services.

This paper presents an overview of the BIIRRR 2019 workshop at CHIIR 2019, which had the explicit aim of understanding and promoting re-use of resources for interactive IR experimentation.

Open AccessOpportunities and Risks in Digital Humanities ResearchMark HallMark Hallhttps://doi.org/10.14220/9783737011778.47SectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinkedInRedditEmail About Previous chapter Next chapter FiguresReferencesRelatedDetailsCited byText analysis using deep neural networks in digital humanities and information science30 June 2021 | Journal of the Association for Information Science and Technology, Vol. 2 Download book coverDH&CSVolume 1 1. AuflageISBN: 978-3-8471-1177-1 eISBN: 978-3-7370-1177-8HistoryPublished online:June 2020 Information© 2020, V&R unipress GmbH, Theaterstraße 13, 37073 Göttingen, GermanyDieses Werk ist als Open-Access-Publikation im Sinne der Creative-Commons-Lizenz BY-SA International 4.0 ("Namensnennung – Weitergabe unter gleichen Bedingungen") unter dem DOI 10.14220/9783737011778 abzurufen. Um eine Kopie dieser Lizenz zu sehen, besuchen Sie https://creativecommons.org/licenses/by-sa/4.0/.PDF download

Museums are increasing access to their collections and providing richer user experiences via web-based interfaces. However, they are seeing high numbers of users looking at only one or two pages within 10 s and then leaving. To reduce this rate, a better understanding of the type of user who visits a museum website is required. Existing models for museum website users tend to focus on groups that are readily accessible for study or provide little detail in their definitions of the groups. This paper presents the results of a large-scale user survey for the National Museums Liverpool museum website in which data on a wide range of user characteristics were collected regarding their current visit to provide a better understanding of their motivations, tasks, engagement and domain knowledge. Results show that the frequently understudied general public and non-professional users make up the majority (approximately 77%) of the respondents.

Increasing re-use in Interactive Information Retrieval (IIR) has been an ongoing aim in IIR for a significant amount of time, however progress has been limited and patchy. While re-use of some study aspects can be difficult due to the varied nature of IIR studies, the use of pre- and post-task self-reported measures is widespread and relatively standardised. Nevertheless, re-use of elements in this area is also limited, in part because systems used to implement them are not able to exchange question, instruments, or complete study setups. To address this, this paper presents a standardised, but extendable, format for IIR survey instrument exchange.

Users looking for books online are confronted with both professional meta-data and user-generated content. The goal of the Interactive Social Book Search Track was to investigate how users used these two sources of information, when looking for books in a leisure context. To this end, participants recruited by seven teams performed two main tasks and one optional task using a user-interface that supports multiple search stages.

Users looking for books online are confronted with both professional meta-data and user-generated content. The goal of the Interactive Social Book Search Track was to investigate how users used these two sources of information, when looking for books in a leisure context. To this end, participants recruited by seven teams performed two main tasks and one optional task using a user-interface that supports multiple search stages.