In recent years, many trust-based social recommender systems have focused on heterogeneity in trust relations, but this heterogeneity is considered only for explicit neighbors. Most of the existing works overlook heterogeneity in the high-order network structure of user-user social networks. Most of them assume that the power of trust relationship of a neighbor on a target user is constant and applies the same value when considering different categories of recommendations. To overcome the above challenges, we propose a model architecture named Motif-induced attention-based Capsule Graph Convolutional Networks (mCGCNs). To the best of our knowledge, this is the first study in which the capsule network extracts multiple latent social-aware user vectors and latent preference-based user vectors of a target user, which vary from item to item based on the recommendation. To extract multiple latent social-aware user vectors, we not only consider explicit neighbors but also consider implicit neighbors, and regarding this the motif networks capture the complex higher-level pattern of interactivities among users. The investigations and empirical analyses on publicly available real-world datasets (Ciao, Epinions, and Library Thing) illustrate the effectiveness of our model compared to 13 popular baselines. It outperforms the best baseline model by margins ranging from 9.86% to 12.78% in HR@10, 13.07% to 13.55% in NDCG@10, 7.63% to 8.97% in MAE, 6.05% to 7.24% in RMSE for three datasets of product recommendations. Through ablation study, key components in mCGCNs are validated to benefit the recommendation performance improvement.

In recent years, advances in natural language processing (NLP) have increasingly relied on computational infrastructure, including hardware accelerators, scalable memory systems, software libraries, and framework, and widespread adoption of cloud platforms. However, existing entity recognition methods and scientific knowledge graphs largely overlook these components, instead focusing on research tasks, methods, datasets, and evaluation metrics. To address this gap, we present InfraKG, a large-scale Infrastructure Knowledge Graph that captures and links infrastructure-related entities mentioned in scientific publications. InfraKG is built using a hybrid information extraction framework applied to 85,000 arXiv papers in the computational linguistics domain, combining transformer-based NER models, semantic sentence filtering, and large language models (LLMs). The resulting graph contains 166,728 nodes and 1.5 million relations across seven types, connecting infrastructure entities to scientific publications along with their metadata. InfraKG is the first large-scale resource to systematically represent computational infrastructure in NLP research, enabling advanced queries, trend analysis, and infrastructure-aware literature reviews. We evaluated the proposed framework on 470 manually annotated PDF papers for infrastructure entities, covering a test set of 20,774 sentences. All code and data are publicly available at: code repository .

Today's computer applications are able to store large amounts of knowledge. In addition to large language models (LLMs), such as ChatGPT, knowledge graphs, which store facts in the form of triples, are also used in modern information and knowledge systems. This summary of a lecture provides an introduction to these technologies and uses examples to show how they work and what their strengths and weaknesses are.

Information retrieval systems for the patent domain have a long and evolving history, serving as effective tools to support patent experts in a variety of daily tasks. They facilitate patent landscape analysis, help in the drafting and evaluation tasks in the patenting process, and enable efficient information extraction to gain practical insights into new technologies and innovations. Moreover, they assist in identifying existing solutions, knowledge gaps, trends, and persistent challenges within specific technological fields, thereby informing strategic decision-making and innovation management. Advances in machine learning and natural language processing allow to further automate such tasks, e.g. paragraph retrieval, question answering (QA) or patent text generation. The exploration of semantic technologies for the intellectual property (IP) industry is still in its early stages, with significant potential yet to be unlocked. Investigating the use of artificial intelligence (AI) methods for the patent domain is therefore not only of academic interest, but also highly relevant for practitioners. Compared to other domains, high quality, semi-structured, annotated data is available in large volumes (a requirement for supervised machine learning models), making training large models easier. On the other hand, domain-specific challenges arise, such as very technical language or legal requirements for patent documents, and data from various disciplines and technological areas. With the 6th edition of this workshop we will provide a platform for researchers and industry to discuss recent developments for semantic patent retrieval and analysis employing sophisticated methods ranging from patent text mining, domain-specific information retrieval to large language models (LLMs) targeting next generation applications and use cases for the IP and related domains.

In recent years, pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers) have demonstrated exceptional performance across various natural language processing tasks. However, its effectiveness of encoding and capturing fine-grained distinctions within the hidden latent space during fine-tuning on coarse-grained labels remains relatively unexplored. To investigate this, we performed two distinct tasks: clustering and few-shot classification on fine-grained labels. The representations extracted from BERT's hidden layers are utilized as input for these tasks. In the few-shot classification task, we demonstrate that the BERT model encodes valuable information about fine-grained labels during its fine-tuning on coarse-grained labels, allowing the few-shot classifier to classify fine-grained classes accurately even with a limited number of data samples. Additionally, in the clustering analysis, a thorough examination of the hidden layers is conducted to identify clusters that align with fine-grained label distinctions. The identification of such patterns further proves that the BERT model indeed encodes fine-grained label information within its hidden layers even when fine-tuned on coarse-grained labels. The findings contribute to a deeper understanding of the capabilities of the BERT model and provide valuable insights into harnessing its hidden latent space for fine-grained classification tasks.

Access to reliable scientific knowledge is crucial to making informed decisions for both policymakers and citizens. However, scientific documents are inherently difficult due to their complex terminology and vernacular. Automatic text simplification aims to remove some of these barriers. Evaluation frameworks, which include collections and evaluation measures, are designed to assess the generated text simplifications. In this paper, we perform a comparative analysis of current text simplification evaluation measures on both scientific text and a generic corpus based on Wikipedia. Our main finding is that the currently existing measures tend to perform worse on scientific texts and on longer texts consisting of several sentences. More generally, our analysis informs the development of suitable text simplification evaluation measures for scientific texts.

Traditional named entity linking (NEL) tools have largely employed a general-domain approach, spanning across various entity types such as persons, organizations, locations, and events in a multitude of contexts. While multimodal entity linking datasets exist (e.g., disambiguation of person names with the help of photographs), there is a need to develop domain-specific resources that represent the unique challenges present in domains like cultural heritage (e.g., stylistic changes through time, diversity of social and political context). To address this gap, our work presents a novel multimodal entity linking benchmark dataset for the art domain together with a comprehensive experimental evaluation of existing NEL methods on this new dataset. The dataset encapsulates various entities unique to the art domain. During the dataset creation process, we also adopt manual human evaluation, providing high-quality labels for our dataset. We introduce an automated process that facilitates the generation of this art dataset, harnessing data from multiple sources (Artpedia, Wikidata and Wikimedia Commons) to ensure its reliability and comprehensiveness. Furthermore, our paper delineates best practices for the integration of art datasets, and presents a detailed performance analysis of general-domain entity linking systems, when applied to domain-specific datasets. Through our research, we aim to address the lack of datasets for NEL in the art domain, providing resources for the development of new, more nuanced, and contextually rich entity linking methods in the realm of art and cultural heritage.

Transformer-encoder architectures for language modeling provide rich contextualized vectors, representing both, syntactic and semantic information captured during pre-training. These vectors are useful for multiple downstream tasks, but directly using the final layer representations might hide interesting elements represented in the hidden layers. In this paper, we propose Shact Syntactic Hierarchical Agglomerative Clustering from Transformer-Encoders, a model that disentangles syntactic span representations from these hidden representations, into a latent vector space. In our model, spans are expressed in terms of token distances. We propose a loss function that optimizes the neural disentanglement model from ground truth spans, and we propose to integrate these latent space vectors into a two-phase model via hierarchical clustering, suitable for multiple span recognition tasks.We evaluated our approach on flat and nested named entity recognition as well as chunking, showing the model's ability to discover these spans, as well as having competitive results on the full recognition and classification tasks.

Information retrieval systems for the patent domain have a long history. They can support patent experts in a variety of daily tasks: from analyzing the patent landscape to support experts in the patenting process and large-scale information extraction. Advances in machine learning and natural language processing allow to further automate tasks, such as paragraph retrieval, question answering (QA) or even patent text generation. Uncovering the potential of semantic technologies for the intellectual property (IP) industry is just getting started. Investigating the use of artificial intelligence methods for the patent domain is therefore not only of academic interest, but also highly relevant for practitioners. Compared to other domains, high quality, semi-structured, annotated data is available in large volumes (a requirement for supervised machine learning models), making training large models easier. On the other hand, domain-specific challenges arise, such as very technical language or legal requirements for patent documents. With the 5th edition of this workshop we will provide a platform for researchers and industry to learn about novel and emerging technologies for semantic patent retrieval and big analytics employing sophisticated methods ranging from patent text mining, domain-specific information retrieval to large language models targeting next generation applications and use cases for the IP and related domains.

The multi-label automatic classification of scientific publications based on a pre-defined taxonomy, also called automatic subject indexing is a continuing research endeavor with significant cross-domain applicability. In this paper, we assess the performance of X-transformer and its variants with other extreme multi-label classification models for the above task. Our model Weak X-transformer achieves a micro F1-score of 0.65 and 64

In recent years, Explainable AI (xAI) attracted a lot of attention as various countries turned explanations into a legal right. xAI algorithms enable humans to understand the underlying models and explain their behavior, leading to insights through which the models can be analyzed and improved beyond the accuracy metric by, e.g., debugging the learned pattern and reducing unwanted biases. However, the widespread use of xAI and the rapidly growing body of published research in xAI have brought new challenges. A large number of xAI algorithms can be overwhelming and make it difficult for practitioners to choose the correct xAI algorithm for their specific use case. This problem is further exacerbated by the different approaches used to assess novel xAI algorithms, making it difficult to compare them to existing methods. To address this problem, we introduce Compare-xAI, a benchmark that allows for a direct comparison of popular xAI algorithms with a variety of different use cases. We propose a scoring protocol employing a range of functional tests from the literature, each targeting a specific end-user requirement in explaining a model. To make the benchmark results easily accessible, we group the tests into four categories (fidelity, fragility, stability, and stress tests). We present results for 13 xAI algorithms based on 11 functional tests. After analyzing the findings, we derive potential solutions for data science practitioners as workarounds to the found practical limitations. Finally, Compare-xAI is a tentative to unify systematic evaluation and comparison methods for xAI algorithms with a focus on the end-user’s requirements. The code is made available at: https://karim-53.github.io/cxai/ .

As machine learning techniques are being increasingly employed for text processing tasks, the need for training data has become a major bottleneck for their application. Manual generation of large scale training datasets tailored to each task is a time consuming and expensive process, which necessitates their automated generation. In this work, we turn our attention towards creation of training datasets for named entity recognition (NER) in the context of the cultural heritage domain. NER plays an important role in many natural language processing systems. Most NER systems are typically limited to a few common named entity types, such as person, location, and organization. However, for cultural heritage resources, such as digitized art archives, the recognition of fine-grained entity types such as titles of artworks is of high importance. Current state of the art tools are unable to adequately identify artwork titles due to unavailability of relevant training datasets. We analyse the particular difficulties presented by this domain and motivate the need for quality annotations to train machine learning models for identification of artwork titles. We present a framework with heuristic based approach to create high-quality training data by leveraging existing cultural heritage resources from knowledge bases such as Wikidata. Experimental evaluation shows significant improvement over the baseline for NER performance for artwork titles when models are trained on the dataset generated using our framework.

Information retrieval systems for the patent domain have a long history. They can support patent experts in a variety of daily tasks: from analyzing the patent landscape to support experts in the patenting process and large-scale information extraction. Advances in machine learning and natural language processing allow to further automate tasks, such as paragraph retrieval or even patent text generation. Uncovering the potential of semantic technologies for the intellectual property (IP) industry is just getting started. Investigating the use of artificial intelligence methods for the patent domain is therefore not only of academic interest, but also highly relevant for practitioners. Compared to other domains, high quality, semi-structured, annotated data is available in large volumes (a requirement for supervised machine learning models), making training large models easier. On the other hand, domain-specific challenges arise, such as very technical language or legal requirements for patent documents. The focus of the 4th edition of this workshop will be on two-way communication between industry and academia from all areas of information retrieval in particular with the Asian community. We want to bring together novel research results and the latest systems and methods employed by practitioners in the field.

Ultrafine named entity typing (UFET) refers to the assignment of predefined labels to entity mentions in a given context. In contrast to traditional named entity typing, the number of potential labels is in the thousands and one mention can have more than one assigned type. Previous approaches either depend on large training datasets, or require inefficient encoding of all input-type combinations. Therefore, there is a need for investigating the efficiency during training and prediction of entity typing models in the ultrafine-grained setting, considering its distinctively bigger search space, compared to the coarse- and fine-grained tasks. To efficiently solve UFET, we propose Decent, a lightweight model that encodes, using a pretrained language model, the input sentences separately from the type labels. Additionally, we make use of negative oversampling to speed up the training while improving the generalization of unseen types. Using an openly available UFET dataset, we evaluated the classification and runtime performance of Decent and observed that training and prediction runtime is orders of magnitude faster than the current state-of-the-art approaches, while maintaining a competitive classification performance.

Knowledge graphs (KGs) provide structured representation of data in the form of relations between different entities. The semantics of relations between words and entities are often ambiguous, where it is common to find polysemous relations that represent multiple semantics based on the context. This ambiguity in relation semantics also proliferates KG triples. While the guidance from custom-designed ontologies addresses this issue to some extent, our analysis shows that the heterogeneity and complexity of real-world data still results in substantial relation polysemy within popular KGs. The correct semantic interpretation of KG relations is necessary for many downstream applications such as entity classification and question answering. We present the problem of fine-grained relation discovery and a data-driven method towards this task that leverages the vector representations of the knowledge graph entities and relations available from relational learning models. We show that by performing clustering over these vectors, our method is able to not only identify the polysemous relations in knowledge graphs, but also discover the different semantics associated with them. Extensive empirical evaluation shows that fine-grained relations discovered by the proposed approach lead to substantial improvement in the semantics in the Yago and NELL datasets, as compared to baselines. Additional insights from qualitative analyses convey that fine-grained relation discovery is an important yet complex task, especially in the presence of complex ontologies and noisy data.

In recent years, Explainable AI (xAI) attracted a lot of attention as various countries turned explanations into a legal right. xAI allows for improving models beyond the accuracy metric by, e.g., debugging the learned pattern and demystifying the AI's behavior. The widespread use of xAI brought new challenges. On the one hand, the number of published xAI algorithms underwent a boom, and it became difficult for practitioners to select the right tool. On the other hand, some experiments did highlight how easy data scientists could misuse xAI algorithms and misinterpret their results. To tackle the issue of comparing and correctly using feature importance xAI algorithms, we propose Compare-xAI, a benchmark that unifies all exclusive functional testing methods applied to xAI algorithms. We propose a selection protocol to shortlist non-redundant functional tests from the literature, i.e., each targeting a specific end-user requirement in explaining a model. The benchmark encapsulates the complexity of evaluating xAI methods into a hierarchical scoring of three levels, namely, targeting three end-user groups: researchers, practitioners, and laymen in xAI. The most detailed level provides one score per test. The second level regroups tests into five categories (fidelity, fragility, stability, simplicity, and stress tests). The last level is the aggregated comprehensibility score, which encapsulates the ease of correctly interpreting the algorithm's output in one easy to compare value. Compare-xAI's interactive user interface helps mitigate errors in interpreting xAI results by quickly listing the recommended xAI solutions for each ML task and their current limitations. The benchmark is made available at https://karim-53.github.io/cxai/