Root-Cause Analysis (RCA) seeks to identify the variables responsible for abnormal system behavior in complex domains such as manufacturing, cloud computing, and healthcare. Existing approaches face a critical bottleneck: graph-based causal methods can identify intervention targets but typically require a known or accurately estimated causal graph, while graph-free statistical methods either localize marginal anomalies rather than structural causes, or rely on restrictive assumptions about graph structure or functional form. We propose StableRCA, a local mechanism-level RCA framework that avoids global graph discovery by estimating local Markov boundaries and detecting conditional distribution shifts within them. Leveraging the Independent Causal Mechanism principle, we show that intervention targets can be identified with probability converging exponentially in sample size under faithful Markov boundary recovery and non-degenerate mechanism shifts. Experiments on synthetic benchmarks and five real-world datasets demonstrate that StableRCA is robust to graph misspecification, effective under multiple intervention targets, scalable to large systems, and reliable across diverse application domains. Code is available at: https://anonymous.4open.science/r/StableRCA-E362

Causal analysis is a crucial task in many domains, including manufacturing, social science, and medicine. However, despite recent progress, the conceptual and methodological complexity of causal methods makes them largely inaccessible to domain experts. This gap prevents experts from leveraging these advances and hinders researchers who lack access to real-world data for validation. To bridge this divide, we introduce ORCA, a copilot for end-to-end causal analysis. ORCA orchestrates agents to understand the user's goals and guide them through the most appropriate causal analysis workflow, from fully automatic to highly user-guided execution. It features causal discovery, causal effect estimation, explainability and Root-Cause-Analysis (RCA). ORCA evaluates and compares performance, generates key metrics and diagrams, and generates insights through structured reports. We highlight its effectiveness across several real-world use-cases.

Autonomous driving requires accurate scene understanding, including road geometry, traffic agents, and their semantic relationships. In online HD map generation scenarios, raster-based representations are well-suited to vision models but lack geometric precision, while graph-based representations retain structural detail but become unstable without precise maps. To harness the complementary strengths of both, we propose DiffSemanticFusion-a fusion framework for multimodal trajectory prediction and planning. Our approach reasons over a semantic raster-fused BEV space, enhanced by a map diffusion module that improves both the stability and expressiveness of online HD map representations. We validate our framework on two downstream tasks: trajectory prediction and planning-oriented end-to-end autonomous driving. Experiments on real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate improved performance over several state-of-the-art (SOTA) methods. For the prediction task on nuScenes, we integrate DiffSemanticFusion with the online HD map informed QCNet, achieving a 5.1% performance improvement. For end-to-end autonomous driving in NAVSIM, DiffSemanticFusion achieves SOTA results, with a 15% performance gain in NavHard scenarios. In addition, extensive ablation and sensitivity studies show that our map diffusion module can be seamlessly integrated into other vector-based approaches to enhance performance.

A comprehensive understanding of causality is critical for navigating and operating within today's complex real-world systems. The absence of realistic causal models with known data generating processes complicates fair benchmarking. In this paper, we present the CausalMan simulator, modeled after a real-world production line. The simulator features a diverse range of linear and non-linear mechanisms and challenging-to-predict behaviors, such as discrete mode changes. We demonstrate the inadequacy of many state-of-the-art approaches and analyze the significant differences in their performance and tractability, both in terms of runtime and memory complexity. As a contribution, we will release the CausalMan large-scale simulator. We present two derived datasets, and perform an extensive evaluation of both.

Trajectory prediction in autonomous driving relies on accurate representation of all relevant contexts of the driving scene, including traffic participants, road topology, traffic signs, as well as their semantic relations to each other. Despite increased attention to this issue, most approaches in trajectory prediction do not consider all of these factors sufficiently. We present SemanticFormer, an approach for predicting multimodal trajectories by reasoning over a semantic traffic scene graph using a hybrid approach. It utilizes high-level information in the form of meta-paths, i.e. trajectories on which an agent is allowed to drive from a knowledge graph which is then processed by a novel pipeline based on multiple attention mechanisms to predict accurate trajectories. SemanticFormer comprises a hierarchical heterogeneous graph encoder to capture spatio-temporal and relational information across agents as well as between agents and road elements. Further, it includes a predictor to fuse different encodings and decode trajectories with probabilities. Finally, a refinement module assesses permitted meta-paths of trajectories and speed profiles to obtain final predicted trajectories. Evaluation of the nuScenes benchmark demonstrates improved performance compared to several SOTA methods. In addition, we demonstrate that our knowledge graph can be easily added to two graph-based existing SOTA methods, namely VectorNet and LaFormer, replacing their original homogeneous graphs. The evaluation results suggest that by adding our knowledge graph the performance of the original methods is enhanced by 5% and 4%, respectively.

Accurate trajectory prediction is crucial for ensuring safe and efficient autonomous driving. However, most existing methods overlook complex interactions between traffic participants that often govern their future trajectories. In this paper, we propose SocialFormer, an agent interaction-aware trajectory prediction method that leverages the semantic relationship between the target vehicle and surrounding vehicles by making use of the road topology. We also introduce an edge-enhanced heterogeneous graph transformer (EHGT) as the aggregator in a graph neural network (GNN) to encode the semantic and spatial agent interaction information. Additionally, we introduce a temporal encoder based on gated recurrent units (GRU) to model the temporal social behavior of agent movements. Finally, we present an information fusion framework that integrates agent encoding, lane encoding, and agent interaction encoding for a holistic representation of the traffic scene. We evaluate SocialFormer for the trajectory prediction task on the popular nuScenes benchmark and achieve state-of-the-art performance.

Trajectory prediction in traffic scenes involves accurately forecasting the behaviour of surrounding vehicles. To achieve this objective it is crucial to consider contextual information, including the driving path of vehicles, road topology, lane dividers, and traffic rules. Although studies demonstrated the potential of leveraging heterogeneous context for improving trajectory prediction, state-of-the-art deep learning approaches still rely on a limited subset of this information. This is mainly due to the limited availability of comprehensive representations. This paper presents an approach that utilizes knowledge graphs to model the diverse entities and their semantic connections within traffic scenes. Further, we present nuScenes Knowledge Graph (nSKG), a knowledge graph for the nuScenes dataset, that models explicitly all scene participants and road elements, as well as their semantic and spatial relationships. To facilitate the usage of the nSKG via graph neural networks for trajectory prediction, we provide the data in a format, ready-to-use by the PyGlibrary. All artefacts can be found here: https://tinyurl.com/5t2vv9yu.

Motion prediction and planning are key components to enable autonomous driving. Although high definition (HD) maps provide important contextual information that constrains the action space of traffic participants, most approaches are not able to fully exploit this heterogeneous information. In this work, we enrich the existing road geometry of the popular nuScenes dataset and convert it into the open-source map framework Lanelet2. This allows easy access to the road topology and thus, enables the usage of (1) spatial semantic information, such as agents driving on intersecting roads and (2) map-generated anchor paths for target vehicles that can help to improve trajectory prediction performance. Further, we present DMAP, a simple, yet effective approach for diverse map-based anchor path generation and filtering. We show that combining DMAP with ground truth velocity profile information yields high-quality motion prediction results on nuScenes (MinADE 5 =1.09, MissRate 5,2 =0.18, Offroad rate=0.00). While it is obviously unfair to compare us against the state-of-the-art, it shows that our HD map accurately depicts the road geometry and topology. Future approaches can leverage this by focusing on data-driven sampling of map-based anchor paths and estimating velocity profiles. Moreover, our HD map can be used for map construction tasks and supplement perception. Code and data are made publicly available at https://felixhertlein.github.io/lanelet4nuscenes.

Autonomous Driving (AD) datasets, when used in combination with deep learning techniques, have enabled significant progress on difficult AD tasks such as perception, trajectory prediction, and motion planning. These datasets represent the content of driving scenes as captured by various sensors, including cameras, RADAR, and LiDAR, along with 2D/3D annotations of traffic participants. Such datasets, however, often fail to capture and to represent the spatial, temporal, and semantic relations between entities in a scene. This lack of knowledge leads to a shallow understanding of the true complexity and dynamics inherent in a driving scene. In this paper, we argue that a Knowledge Graph (KG)-based representation of driving scenes, that provides a richer structure and semantics, will lead to further improvements in AD. Towards this goal, we developed a layered architecture and ontologies for specific AD datasets and a fundamental ontology of shared concepts. We also built KGs for three different AD datasets. We perform an analysis with respect to information contained in the AD KGs and outline how the additional semantic information contained in the KGs could improve the performance of different AD tasks. Moreover, example queries are provided to retrieve relevant information that can be exploited for augmenting the AD pipelines. All artifacts needed for reproducibility purposes are provided via a GitHub repository. ( https://github.com/boschresearch/dskg-constructor — Note that we removed our internal namespaces of reused ontologies, because of confidentiality and to provide self-contained ontologies. As the original datasets are under specific licences, the KGs are not published, but we provide the scripts to generate them.)

Precisely predicting the future trajectories of surrounding traffic participants is a crucial but challenging problem in autonomous driving, due to complex interactions between traffic agents, map context and traffic rules. Vector-based approaches have recently shown to achieve among the best performances on trajectory prediction benchmarks. These methods model simple interactions between traffic agents but don't distinguish between relation-type and attributes like their distance along the road. Furthermore, they represent lanes only by sequences of vectors representing center lines and ignore context information like lane dividers and other road elements. We present a novel approach for vector-based trajectory prediction that addresses these shortcomings by leveraging three crucial sources of information: First, we model interactions between traffic agents by a semantic scene graph, that accounts for the nature and important features of their relation. Second, we extract agent-centric image-based map features to model the local map context. Finally, we generate anchor paths to enforce the policy in multi-modal prediction to permitted trajectories only. Each of these three enhancements shows advantages over the baseline model HoliGraph.

Representing relevant information of a traffic scene and understanding its environment is crucial for the success of autonomous driving. Modeling the surrounding of an autonomous car using semantic relations, i.e., how different traffic participants relate in the context of traffic rule based behaviors, is hardly been considered in previous work. This stems from the fact that these relations are hard to extract from real-world traffic scenes. In this work, we model traffic scenes in a form of spatial semantic scene graphs for various different predictions about the traffic participants, e.g., acceleration and deceleration. Our learning and inference approach uses Graph Neural Networks (GNNs) and shows that incorporating explicit information about the spatial semantic relations between traffic participants improves the predicdtion results. Specifically, the acceleration prediction of traffic participants is improved by up to 12% compared to the baselines, which do not exploit this explicit information. Furthermore, by including additional information about previous scenes, we achieve 73% improvements.

Automated driving is one of the most active research areas in computer science. Deep learning methods have made remarkable breakthroughs in machine learning in general and in automated driving (AD)in particular. However, there are still unsolved problems to guarantee reliability and safety of automated systems, especially to effectively incorporate all available information and knowledge in the driving task. Knowledge graphs (KG) have recently gained significant attention from both industry and academia for applications that benefit by exploiting structured, dynamic, and relational data. The complexity of graph-structured data with complex relationships and inter-dependencies between objects has posed significant challenges to existing machine learning algorithms. However, recent progress in knowledge graph embeddings and graph neural networks allows to applying machine learning to graph-structured data. Therefore, we motivate and discuss the potential benefit of KGs applied to the main tasks of AD including 1) ontologies 2) perception, 3) scene understanding, 4) motion planning, and 5) validation. Then, we survey, analyze and categorize ontologies and KG-based approaches for AD. We discuss current research challenges and propose promising future research directions for KG-based solutions for AD.

Automated Driving (AD) datasets, when used in combination with deep learning techniques, have enabled significant progress on difficult AD tasks such as perception, trajectory prediction and motion planning. These datasets represent the content of driving scenes as captured by various sensors, including cameras, RADAR, and LiDAR, along with 2D/3D annotations of traffic participants. Such datasets, however, often fail to capture and to represent the spatial, temporal, functional, and semantic relations between entities in a scene. This lack of knowledge leads to a shallow understanding of the true complexity and dynamics inherent in a driving scene. In this paper, we argue that a knowledge graph based representation of driving scenes, that provides a richer structure and semantics, will lead to further improvements in automated driving. Towards this goal, we developed a layered architecture and ontologies for specific automated driving datasets and a fundamental ontology of shared concepts. We also built knowledge graphs (KG) for three different AD datasets. We perform an analysis w.r.t. information contained in the AD KGs and outline how the additional semantic information contained in the KGs could improve the performance of different AD tasks. Moreover, example queries are provided to retrieve relevant information that can be exploited for augmenting the AD pipelines. All artefacts needed for reproducability purposes are provided via a Dropbox folder 1 1 shorturl.at/iwyCV - we will go through an internal approval process for making all artefacts publicly available. We removed our internal namespaces of reused ontologies, because of confidentiality and to provide self-contained ontologies. As the original datasets are under specific licences we can not publish the KGs themselves, but we provided the scripts to generate them.

Context-aware Recommender Systems (CARS) are becoming an integral part of the everyday life by providing users the ability to retrieve relevant information based on their contextual situation. To increase the predictive power considering many parameters, such as mood, hunger level and user preferences, information from heterogeneous sources should be leveraged. However, these data sources are typically isolated and unexplored and the efforts for integrating them are exacerbated by variety of data structures used for their modelling and costly pre-processing operations. We propose a Knowledge Graph-based approach to allow integration of data according to abstract semantic models for Points-of-Interests (POI)s recommendation scenarios. By enriching data with information about attributes, relationships and their meaning, additional knowledge can be derived from what already exists. We demonstrate the applicability of the proposed approach with a concrete example showing benefits of the retrieving the dispersed data with a unified access mechanism.

The task of safe driving poses a huge challenge for drivers in day to day driving situations. Many times, this task can be very difficult, e.g., due to dense traffic, bad weather conditions, or a risky driving maneuver, and thus demand high concentration of the driver. The difficulty level escalates by the ever-increasing infotainment offers inside vehicles or distractions caused by occupants thus making substantial contribution to the driver distraction. This often results in dangerous driving situations which could be avoided by Advanced Driver Assistance Systems or highly automated driving systems taking the situation difficulty into account. E.g., an incoming phone call is postponed during a difficult situation. However, current systems do not consider all factors that influence the difficulty of a given situation. In this paper, we present an approach for estimating the difficulty of a driving situation by combining a number of different factors, such as environmental, insidevehicle, driver state and personal characteristics, respectively. Our approach follows a rule-based paradigm to make the difficulty estimation reproducible and adjustable to current traffic rules. It is based on a generic and modularized architecture to allow integration and abstraction from heterogeneous data sources. Further, a feedback is provided to the driver or system to explain the contribution of the various factors to the difficulty status. Finally, we demonstrate the capability of the proposed approach with concrete examples, where we estimate the difficulty in various driving scenarios and for different drivers.

Learned latent vector representations are key to the success of many recommender systems in recent years. However, traditional approaches like matrix factorization produce vector representations that capture global distributions of a static recommendation scenario only. Such latent user or item representations do not capture background knowledge and are not customized to a concrete situational context and the sequential history of events leading up to it. This is a fundamentally limiting restriction for many tasks and applications, since the latent state can depend on a) abstract background information, b) the current situational context and c) the history of related observations. An illustrating example is a restaurant recommendation scenario, where a user’s assessment of the situation depends a) on taxonomical information regarding the type of cuisine, b) on situational factors like time of day, weather or location and c) on the subjective individual history and experience of this user in preceding situations. This situation-specific internal state of the user is not captured when using a traditional collaborative filtering approach, since background knowledge, the situational context and the sequential nature of an individual’s history cannot easily be represented in the matrix. In this paper, we investigate how well state-of-the-art approaches do exploit those different dimensions relevant to POI recommendation tasks. Naturally, we represent such a scenario as a temporal knowledge graph and compare plain knowledge graph, a taxonomy and a hypergraph embedding approach, as well as a recurrent neural network architecture to exploit the different context-dimensions of such rich information. Our empirical evidence indicates that the situational context is most crucial to the prediction performance, while the taxonomical and sequential information are harder to exploit. However, they still have their specific merits depending on the situation.

Future in-vehicle recommendation systems will assist the driver or passenger in all situations before, along, and after a trip. Based on preferences and needs of the user and by taking the current situation and available context information into account, they will provide the right recommendation at the right time. Bosch is the world's largest automotive supplier, delivering a full range of products and services from power-train, infotainment, HMI, connected mobility, driver assistance to automated driving. This talk will present challenges, concepts and recent technical progress in in-vehicle recommendation systems developed at Bosch including details of a combined routing, charging, and point-of-interest (POI) recommendation system. There has been tremendous progress in the field of location-independent recommendation systems, such as recommending films, music, news or shopping articles. The ubiquity of user location information, provided by connected devices, has paved the way for location-based services (LBS), and their combination with social networks have extended these to location-based social network (LBSN) services, see [1, 6] for recent surveys about recommender systems in LBSN. In-vehicle recommendation systems go a step further by extending LBSN services with vehicle context and vehicle specific applications. This can support the user in various applications, such as routing (e.g. route and point of interest recommendation), infotainment (e.g. music or news recommendation), communication (finding a contact, fast call) and in-vehicle control (e.g. seat position, ambient light or HVAC settings). Out-of-vehicle assistance includes the control of connected devices in smart buildings such as alarm systems, heating, kitchen and entertainment devices. We present an important application of in-vehicle recommending systems, a combined routing, charging and POI recommender developed at Bosch. Routing and charging optimization for electric vehicles was described for optimizing the shortest feasible path [2], optimizing constrained shortest path [4], optimizing charging grid demand and opportunities [5], and optimizing minimum cost [3]. These approaches focus on single criteria based optimization. We describe the first system with combined route optimization, charging station search and POI recommendation. It optimizes three criteria: finding the optimal route with the optimal charging stations, so that the vehicle always has enough energy, and finding the optimal POIs along the route, where 'optimal' depends on the drivers preferences and rich context information covering user, vehicle and environment.

In this paper, we present a new approach for facial face recognition. The method is based on the Fourier transform of Gabor filters and the method of regularized linear discriminate analysis applied to facial features previously localized. The process of facial face recognition is based on two phases: location and recognition. The first phase determines the characteristic using the local properties of the face by the variation of gray level along the axis of the characteristic and the geometric model, and the second phase generates the feature vector by the convolution of the Fourier transform of 40 Gabor filters and face, followed by application of the method of regularized linear discriminate analysis on the vectors of characteristics. Experimental results obtained on sample of images from the XM2VTSDB database [1] have shown that the proposed algorithm gives satisfactory results in a precise manner.

We have made significant progress in automatic speech recognition (ASR) for well-defined applications like dictation and medium vocabulary transaction processing tasks in relatively controlled environments. However, for ASR to approach human levels of performance and for speech to become a truly pervasive user interface, we need novel, nontraditional approaches that have the potential of yielding dramatic ASR improvements. Visual speech is one such source for making large improvements in high...