The goal of our approach is to guide the text prompt construction of anomalous images by leveraging prior knowledge about the anomalies, their characteristics, and co-existence. This information is represented in a Knowledge Graph (KG) comprising classes of various products and anomalies, along with their relationships and the augmentations that can be applied. The ultimate task is to train and evaluate foundation models based on a spatial semantic representation of anomalies and a suitable diffusion model capable of learning forms of anomalies, and generating them for objects of the same class or augmenting them.

Ontology extension refers to the process of enriching an existing ontology in response to emerging requirements, making it more complete. This task is a resource-intensive and error-prone process. Large Language Models (LLMs) have shown promising performance on generating ontologies from scratch, but current approaches rarely tie ontology extension explicitly to requirements or reusable core models, and offer limited, systematic evaluation of LLM outputs. This paper introduces OntoExtend, a requirements-driven framework for ontology extension with LLMs. It uses retrieval-augmented generation (RAG) over relevant input ontologies and requirements in the form of competency questions to propose grounded extensions. We evaluate OntoExtend on 39 CQs from two use cases: a public EU-project ontology, Onto-DESIDE, and an industrial ontology from Bosch. The generated fragments show few structural issues, satisfy all functional evaluation tests, and are rated by ontology engineers as requiring minor to moderate revision before integration. These results suggest that OntoExtend is useful as a drafting assistant for requirement-driven ontology extension in real world scenarios, while remaining sensitive to CQ specificity and modelling profile.

A core challenge data consumers face in large enterprise data lakes is to efficiently understand and reuse data assets due to the prevalence of technical, often vendor-specific, naming conventions. This paper introduces a novel Gen-AI-based co-pilot designed to automate and significantly improve the process of semantic mapping within enterprise data lakes. The proposed solution leverages an agentic workflow powered by Large Language Models (LLMs), a vector DB for similarity search, and a Knowledge Graph (KG) representing Bosch’s internal vocabularies and semantic data models. Empirical evaluation on real-world data demonstrates a sufficiently high accuracy in automatically suggesting correct semantic mappings, leading to significant time savings for data stewards, and enhanced data transparency and interoperability in line with FAIR data principles [2].

The autonomous driving field has seen remarkable advancements in various topics, such as object recognition, trajectory prediction, and motion planning. However, current approaches face limitations in effectively comprehending the complex evolutions of driving scenes over time. This paper proposes FM4SU, a novel methodology for training a symbolic foundation model (FM) for scene understanding in autonomous driving. It leverages knowledge graphs (KGs) to capture sensory observation along with domain knowledge such as road topology, traffic rules, or complex interactions between traffic participants. A bird's eye view (BEV) symbolic representation is extracted from the KG for each driving scene, including the spatio-temporal information among the objects across the scenes. The BEV representation is serialized into a sequence of tokens and given to pre-trained language models (PLMs) for learning an inherent understanding of the co-occurrence among driving scene elements and generating predictions on the next scenes. We conducted a number of experiments using the nuScenes dataset and KG in various scenarios. The results demonstrate that fine-tuned models achieve significantly higher accuracy in all tasks. The fine-tuned T5 model achieved a next scene prediction accuracy of 86.7%. This paper concludes that FM4SU offers a promising foundation for developing more comprehensive models for scene understanding in autonomous driving.

Knowledge graphs (KGs) have transformed data management within the manufacturing industry, offering effective means for integrating disparate data sources through shared and structured conceptual schemas. However, harnessing the power of KGs can be daunting for non-experts, as it often requires formulating complex SPARQL queries to retrieve specific information. With the advent of Large Language Models (LLMs), there is a growing potential to automatically translate natural language queries into the SPARQL format, thus bridging the gap between user-friendly interfaces and the sophisticated architecture of KGs. The challenge remains in adequately informing LLMs about the relevant context and structure of domain-specific KGs, e.g., in manufacturing, to improve the accuracy of generated queries. In this paper, we evaluate multiple strategies that use LLMs as mediators to facilitate information retrieval from KGs. We focus on the manufacturing domain, particularly on the Bosch Line Information System KG and the I40 Core Information Model. In our evaluation, we compare various approaches for feeding relevant context from the KG to the LLM and analyze their proficiency in transforming real-world questions into SPARQL queries. Our findings show that LLMs can significantly improve their performance on generating correct and complete queries when provided only the adequate context of the KG schema. Such context-aware prompting techniques help LLMs to focus on the relevant parts of the ontology and reduce the risk of hallucination. We anticipate that the proposed techniques help LLMs to democratize access to complex data repositories and empower informed decision-making in manufacturing settings.

Motion prediction and planning are key components to enable autonomous driving. Although high definition (HD) maps provide important contextual information that constrains the action space of traffic participants, most approaches are not able to fully exploit this heterogeneous information. In this work, we enrich the existing road geometry of the popular nuScenes dataset and convert it into the open-source map framework Lanelet2. This allows easy access to the road topology and thus, enables the usage of (1) spatial semantic information, such as agents driving on intersecting roads and (2) map-generated anchor paths for target vehicles that can help to improve trajectory prediction performance. Further, we present DMAP, a simple, yet effective approach for diverse map-based anchor path generation and filtering. We show that combining DMAP with ground truth velocity profile information yields high-quality motion prediction results on nuScenes (MinADE 5 =1.09, MissRate 5,2 =0.18, Offroad rate=0.00). While it is obviously unfair to compare us against the state-of-the-art, it shows that our HD map accurately depicts the road geometry and topology. Future approaches can leverage this by focusing on data-driven sampling of map-based anchor paths and estimating velocity profiles. Moreover, our HD map can be used for map construction tasks and supplement perception. Code and data are made publicly available at https://felixhertlein.github.io/lanelet4nuscenes.

The usage of semantic technologies for purposes like data integration, information retrieval, search, and decision-making is steadily on the rise.

Precisely predicting the future trajectories of surrounding traffic participants is a crucial but challenging problem in autonomous driving, due to complex interactions between traffic agents, map context and traffic rules. Vector-based approaches have recently shown to achieve among the best performances on trajectory prediction benchmarks. These methods model simple interactions between traffic agents but don't distinguish between relation-type and attributes like their distance along the road. Furthermore, they represent lanes only by sequences of vectors representing center lines and ignore context information like lane dividers and other road elements. We present a novel approach for vector-based trajectory prediction that addresses these shortcomings by leveraging three crucial sources of information: First, we model interactions between traffic agents by a semantic scene graph, that accounts for the nature and important features of their relation. Second, we extract agent-centric image-based map features to model the local map context. Finally, we generate anchor paths to enforce the policy in multi-modal prediction to permitted trajectories only. Each of these three enhancements shows advantages over the baseline model HoliGraph.

Representing relevant information of a traffic scene and understanding its environment is crucial for the success of autonomous driving. Modeling the surrounding of an autonomous car using semantic relations, i.e., how different traffic participants relate in the context of traffic rule based behaviors, is hardly been considered in previous work. This stems from the fact that these relations are hard to extract from real-world traffic scenes. In this work, we model traffic scenes in a form of spatial semantic scene graphs for various different predictions about the traffic participants, e.g., acceleration and deceleration. Our learning and inference approach uses Graph Neural Networks (GNNs) and shows that incorporating explicit information about the spatial semantic relations between traffic participants improves the predicdtion results. Specifically, the acceleration prediction of traffic participants is improved by up to 12% compared to the baselines, which do not exploit this explicit information. Furthermore, by including additional information about previous scenes, we achieve 73% improvements.

Deep learning techniques achieve high accuracy in computer vision tasks. However, their accuracy suffers considerably when they face a domain change, i.e., as soon as they are used in a domain that differs from their training domain. For example, a road sign recognition model trained to recognize road signs in Germany performs poorly in countries with different road sign standards like China. We propose ConTraKG, a neuro-symbolic approach that enables cross-domain transfer learning based on prior knowledge about the domain or context. A knowledge graph serves as a medium for encoding such prior knowledge, which is then transformed into a dense vector representation via embedding methods. Using a five-phase training pipeline, we train the deep neural network to adjust its visual embedding space according to the domaininvariant embedding space of the knowledge graph based on a contrastive loss function. This allows the neural network to incorporate training data from different target domains that are already represented in the knowledge graph. We conduct a series of empirical evaluations to determine the accuracy of our approach. The results show that ConTraKG is significantly more accurate than the conventional approach for dealing with domain changes. In a transfer learning setup, where the network is trained on both domains, ConTraKG achieves 21% higher accuracy when tested on the source domain and 15% when tested on the target domain compared to the standard approach. Moreover, with only 10% of the target data for training, it achieves the same accuracy as the cross-entropy-based model trained on the full target data.

Data lake architectures enable the storage and retrieval of large amounts of data across an enterprise. At Robert Bosch GmbH, we have deployed a data lake for this expressed purpose, focused on managing automotive sensor data. Simply centralizing and storing data in a data lake, however, does not magically solve critical data management challenges such as data findability, accessibility, interoperability, and re-use. In this paper, we discuss how semantic technologies can help to resolve such challenges. More specifically, we will demonstrate the use of ontologies and knowledge graphs to provide vital data lake functions including the cataloging of data, tracking provenance, access control, and of course semantic search. Of particular importance is the development of the DCPAC Ontology (Data Catalog, Provenance, and Access Control) along with its deployment and use within a large enterprise setting to manage the huge volume and variety of data generated by current and future vehicles.

At the ACMPODC 2020 business meeting and on Zulip, a task force was formed to propose options for changing the PODC/DISC deadline schedule and/or moving to a publication model based on journal-style reviewing. This article summarizes the models identified by the task force and their underlying rationales, and reports on the results of a recent survey among community members, conducted by the task force.

Journal fur LehrerInnenbildung 20 (2020) 1, S. 78-86 Padagogische Teildisziplin: Medienpadagogik; Hochschulforschung und Hochschuldidaktik;

. Machine learning (ML) is gaining much attention for data analysis in manufacturing. Despite the success, there is still a number of challenges in widen-ing the scope of ML adoption. The main challenges include the exhausting effort of data integration and lacking of generalisability of developed ML pipelines to diverse data variants, sources, and domain processes. In this demo we present our SemML system that addresses these challenges by enhancing machine learning with semantic technologies: by capturing domain and ML knowledge in ontologies and ontology templates and automating various ML steps using reasoning. During the demo the attendees will experience three cunningly-designed scenarios based on real industrial applications of manufacturing condition monitoring at Bosch, and witness the power of ontologies and templates in enabling reusable ML pipelines.

. Data management has become a critical ability in today’s data-driven businesses. In the Internet of Things (IoT) domain, sensors, devices and applications generate huge amounts of data. To take advantage of this data, new storage and exchange solutions, e.g., IoT data repositories, enterprise data lakes and IoT data marketplaces, are emerging. These emerging data storage and exchange solutions allow authorized users to discover and access heterogeneous data streams and integrate them across stakeholder boundaries. We present an approach to annotate IoT datasets by mapping their schemata to corresponding ontology terms, paired with an on-the-fly ontology extension mechanism based on templates. We further introduce a framework to evaluate semantic interoperability in this complex setting via the agreement between domain experts on a set of annotation-extension tasks and show how this framework can be used to improve the system iteratively and to avoid potential semantic interoperability conflicts.

In his article Teaching Concurrency Lamport stresses the importance of invariants for the education of engineers and computer students, and presents a short algorithm with a challenge to the reader: find an invariant with which a certain simple property of that distributed algorithm can be proved. Our aim is to compare the invariant proof approach to a different one which uses Tarskian system executions rather than invariants. By comparing the details of the two proofs for this simple algorithm we gain a better understanding of these approaches.

Today, internet of things (IoT) platforms offer proprietary interfaces and protocols. To enable interoperable interaction with those platforms we present the generic BIG IoT API that employs a novel approach for self-description and semantic annotation to fully adapt arbitrary IoT platforms. We have deployed this approach for multiple platforms from the mobility domain.