Safety assurance of Cooperative, Connected, and Automated Mobility (CCAM) systems is a crucial factor for their successful adoption in society, yet it remains a significant challenge. The SUNRISE project has consolidated previous and on-going efforts, and developed a harmonised Safety Assurance Framework (SAF) designed to operationalise the UNECE New Assessment/Test Method (NATM), targeting a wide range of stakeholders including (but not limited to) certifiers, regulators, manufacturers, suppliers, researchers, and assessors. It incorporates a scenario-based approach, underpinned by the system’s Operational Design Domain (ODD) and behaviour for safety assessment. In line with NATM, the SAF consists of multiple pillars: the Audit of manufacturer processes and Safety Management Systems, In-Service Monitoring and Reporting (ISMR) to ensure continued safety during deployment, and Performance Assurance to generate and evaluate safety evidence pre-deployment. While all pillars are integral, this paper concentrates on the Performance Assurance pillar, which integrates three interlinked blocks: Scenario, Environment, and Safety Argument. The Scenario block covers the creation, the formatting, and the storage of logical and concrete scenarios. The Environment block contains an ODD and behaviour based scenario query and retrieval, scenario parameter concretisation, test environment allocation, and test execution. The Safety Argument block contains test evaluation, coverage analysis, safety case formulation, and evaluation decision outcome. Within the SUNRISE project, the SAF has been demonstrated across multiple use cases (various ODDs, systems, and test environment), and several ongoing/ future international collaborative projects are building on top of the SUNRISE SAF and applying it to an even wider set of use cases.

Segmenting objects in an environment is a crucial task for autonomous driving and robotics, as it enables a better understanding of the surroundings of each agent. Although camera sensors provide rich visual details, they are vulnerable to adverse weather conditions. In contrast, radar sensors remain robust under such conditions, but often produce sparse and noisy data. Therefore, a promising approach is to fuse information from both sensors. In this work, we propose a novel framework to enhance camera-only baselines by integrating a diffusion model into a camera-radar fusion architecture. We leverage radar point features to create pseudo-masks using the Segment-Anything model, treating the projected radar points as point prompts. Additionally, we propose a noise reduction unit to denoise these pseudo-masks, which are further used to generate inpainted images that complete the missing information in the original images. Our method improves the camera-only segmentation baseline by 2.63% in mIoU and enhances our camera-radar fusion architecture by 1.48% in mIoU on the Waterscenes dataset. This demonstrates the effectiveness of our approach for semantic segmentation using camera-radar fusion under adverse weather conditions.

Accurate identification of rear light signals in preceding vehicles is pivotal for Advanced Driver Assistance Systems (ADAS), enabling early detection of driver intentions and thereby improving road safety. In this work, we present a novel approach that leverages a meta-learning-enhanced YOLOv8 model to detect left and right turn indicators, as well as brake signals. Traditional radar and LiDAR provide robust geometry, range, and motion cues that can indirectly suggest driver intent (e.g., deceleration or lane drift). However, they do not directly interpret color-coded rear signals, which limits early intent recognition from the taillights. We therefore focus on a camera-based approach that complements ranging sensors by decoding color and spatial patterns in rear lights. This approach to detecting vehicle signals poses additional challenges due to factors such as high reflectivity and the subtle visual differences between directional indicators. We address these by training a YOLOv8 model with a meta-learning strategy, thus enhancing its capability to learn from minimal data and rapidly adapt to new scenarios. Furthermore, we developed a post-processing layer that classifies signals by the geometric properties of detected objects, employing mathematical principles such as distance, area calculation, and Intersection over Union (IoU) metrics. Our approach increases adaptability and performance compared to traditional deep learning techniques, supporting the conclusion that integrating meta-learning into real-time object detection frameworks provides a scalable and robust solution for intelligent vehicle perception, significantly enhancing situational awareness and road safety through reliable prediction of vehicular behavior.

Assuring safety for “AI-based” systems is one of the current challenges in safety engineering. For automated driving systems, in particular, further assurance challenges result from the open context that the systems need to operate in after deployment. The current standardization and regulation landscape for “AI-based” systems is becoming ever more complex, as standards and regulations are being released at high frequencies. This position paper seeks to provide guidance for making qualified arguments which standards should meaningfully be applied to (“AI-based”) automated driving systems. Furthermore, we argue for clearly differentiating sources of risk between AI-specific and general uncertainties related to the open context. In our view, a clear conceptual separation can help to exploit commonalities that can close the gap between system-level and AI-specific safety analyses, while ensuring the required rigor for engineering safe “AI-based” systems.

Hand gesture recognition (HGR) is a convenient and natural form of human–computer interaction. It is suitable for various applications. Much research has already focused on wearable device-based HGR. By contrast, this paper gives an overview focused on device-free HGR. That means we evaluate HGR systems that do not require the user to wear something like a data glove or hold a device. HGR systems are explored regarding technology, hardware, and algorithms. The interconnectedness of timing and power requirements with hardware, pre-processing algorithm, classification, and technology and how they permit more or less granularity, accuracy, and number of gestures is clearly demonstrated. Sensor modalities evaluated are WIFI, vision, radar, mobile networks, and ultrasound. The pre-processing technologies stereo vision, multiple-input multiple-output (MIMO), spectrogram, phased array, range-doppler-map, range-angle-map, doppler-angle-map, and multilateration are explored. Classification approaches with and without ML are studied. Among those with ML, assessed algorithms range from simple tree structures to transformers. All applications are evaluated taking into account their level of integration. This encompasses determining whether the application presented is suitable for edge integration, their real-time capability, whether continuous learning is implemented, which robustness was achieved, whether ML is applied, and the accuracy level. Our survey aims to provide a thorough understanding of the current state of the art in device-free HGR on edge devices and in general. Finally, on the basis of present-day challenges and opportunities in this field, we outline which further research we suggest for HGR improvement. Our goal is to promote the development of efficient and accurate gesture recognition systems.

The ability to accurately detect traffic light color is critical for the functioning of Advanced Driver Assistance Systems (ADAS), as it directly impacts a vehicle’s safety and operational efficiency. This paper introduces Meta-YOLOv8, an improvement over YOLOv8 based on meta-learning, designed explicitly for traffic light color detection focusing on color recognition. In contrast to conventional models, Meta-YOLOv8 focuses on the illuminated portion of traffic signals, enhancing accuracy and extending the detection range in challenging conditions. Furthermore, this approach reduces the computational load by filtering out irrelevant data. An innovative labeling technique has been implemented to address real-time weather-related detection issues, although other bright objects may occasionally confound it. Our model employs meta-learning principles to mitigate confusion and boost confidence in detections. Leveraging task similarity and prior knowledge enhances detection performance across diverse lighting and weather conditions. Meta-learning also reduces the necessity for extensive datasets while maintaining consistent performance and adaptability to novel categories. The optimized feature weighting for precise color differentiation, coupled with reduced latency and computational demands, enables a faster response from the driver and reduces the risk of accidents. This represents a significant advancement for resource-constrained ADAS. A comparative assessment of Meta-YOLOv8 with traditional models, including SSD, Faster R-CNN, and Detection Transformers (DETR), reveals that it outperforms these models, achieving an F1 score, accuracy of 93% and a precision rate of 97%.

Depth estimation remains central to autonomous driving, and radar-camera fusion offers robustness in adverse conditions by providing complementary geometric cues. In this paper, we present XD-RCDepth, a lightweight architecture that reduces the parameters by 29.7

This paper presents a comprehensive impact assessment to explore the potential benefits of harmonized behaviour competencies (BC) for automated driving systems (ADS). Typically, ADS-equipped vehicles operate within certain boundaries specified by an operational design domain (ODD), utilizing the relevant implemented BCs. Nonetheless, many regulatory and standardization-relevant documents employ BC attributes in a non-harmonized manner. The study delves into BC-related activities and applications throughout the entire ADS life cycle, affecting all aspects of the ADS value chain, to gain a deeper understanding of the diverse needs of various stakeholders. BCs are linked to one of the four primary requirement sources at the system level. ADS-related BCs are defined through a multidisciplinary approach driven by their underlying core operating principle: the well-known sense-plan-act cycle. The crucial element within the BC specification is the identified manoeuvre pool, which forms the basis for implementing any route from point A to point B. The individual manoeuvres within the manoeuvre pool are defined by considering the needs of multiple stakeholders. They are based on three essential components: the initial condition, the expected manoeuvre, and the final condition. Furthermore, trustworthy behaviour competencies are specified, encompassing three pillars: robustness, ethics, and lawfulness. Following a detailed stakeholder analysis, several related applications are discussed to highlight the concrete advantages of implementing standardized BCs. The study concludes with a summary of the impact analysis, emphasizing key findings and action points. Lastly, a roadmap is proposed to integrate trustworthy BCs into future ADS. Concretely, the authors developed the following innovations within the scope of this article: (1) Concept for trustworthy behaviour competencies driven by law, ethics, and robustness. (2) Robustness is defined as passenger & ODD awareness and plannable & executable manoeuvre. (3) Manoeuvre pool necessary to implement an arbitrary route from point A to point B. (4) Manoeuvre specification via initial condition, expected behaviour, and final condition. (5) The potential benefits of harmonized behaviour competencies drive impact assessment. (c) 2025 Published by Elsevier Ltd.

This work proposes methodologies for ensuring the trustworthiness of high-risk artificial intelligence (AI) systems (AIS) to achieve compliance with the European Union’s (EU) AI Act. High-risk classified AIS must fulfill seven requirements to be considered trustworthy and human-centric, and subsequently be considered for deployment. These requirements are equally important, mutually supportive, and should be implemented and evaluated throughout the AI lifecycle. The assurance of trustworthiness is influenced by ethical considerations, amongst others. Hence, the operational design domain (ODD) and behavior competency (BC) concepts from the automated driving domain are utilized in risk assessment strategies to quantify different types of residual risks. The methodology presented is guided by the consistent application of the ODD and its related BC concept throughout the entire AI lifecycle, focusing on the trustworthiness assurance framework and its associated process as the main pillars for AIS certification. The achievement of the overall objective of trustworthy and human-centric AIS is divided into seven interconnected sub-goals: the formulation of use restrictions, the trustworthiness assurance/argument itself, the identification of dysfunctional cases, the utilization of scenario databases and datasets, the application of metrics for evaluation, the implementation of the proposed concept across the AI lifecycle, and sufficient consideration of human factors. The role of standards in the assurance process is discussed, considering any existing gaps and areas for improvement. The work concludes with a summary of the developed approach, highlighting key takeaways and action points. Finally, a roadmap to ensure trustworthy and human-centric behavior of future AIS is outlined.

As the number of electronic gadgets in our daily lives is increasing and most of them require some kind of human interaction, this demands innovative, convenient input methods. There are limitations to state-of-the-art (SotA) ultrasound-based hand gesture recognition (HGR) systems in terms of robustness and accuracy. This research presents a novel machine learning (ML)-based end-to-end solution for hand gesture recognition with low-cost micro-electromechanical (MEMS) system ultrasonic transducers. In contrast to prior methods, our ML model processes the raw echo samples directly instead of using pre-processed data. Consequently, the processing flow presented in this work leaves it to the ML model to extract the important information from the echo data. The success of this approach is demonstrated as follows. Four MEMS ultrasonic transducers are placed in three different geometrical arrangements. For each arrangement, different types of ML models are optimized and benchmarked on datasets acquired with the presented custom hardware (HW): convolutional neural networks (CNNs), gated recurrent units (GRUs), long short-term memory (LSTM), vision transformer (ViT), and cross-attention multi-scale vision transformer (CrossViT). The three last-mentioned ML models reached more than 88% accuracy. The most important innovation described in this research paper is that we were able to demonstrate that little pre-processing is necessary to obtain high accuracy in ultrasonic HGR for several arrangements of cost-effective and low-power MEMS ultrasonic transducer arrays. Even the computationally intensive Fourier transform can be omitted. The presented approach is further compared to HGR systems using other sensor types such as vision, WiFi, radar, and state-of-the-art ultrasound-based HGR systems. Direct processing of the sensor signals by a compact model makes ultrasonic hand gesture recognition a true low-cost and power-efficient input method.

Radar and camera fusion yields robustness in perception tasks by leveraging the strength of both sensors. The typical extracted radar point cloud is 2D without height information due to insufficient antennas along the elevation axis, which challenges the network performance. This work introduces a learning-based approach to infer the height of radar points associated with 3D objects. A novel robust regression loss is introduced to address the sparse target challenge. In addition, a multi-task training strategy is employed, emphasizing important features. The average radar absolute height error decreases from 1.69 to 0.25 meters compared to the state-of-the-art height extension method. The estimated target height values are used to preprocess and enrich radar data for downstream perception tasks. Integrating this refined radar information further enhances the performance of existing radar camera fusion models for object detection and depth estimation tasks.

Meta-few-shot learning algorithms, such as Model-Agnostic Meta-Learning (MAML) and Almost No Inner Loop (ANIL), enable machines to learn complex tasks quickly with limited data and based on previous experience. By maintaining the inner loop head of the neural network, ANIL leads to simpler computations and reduces the complexity of MAML. Despite its benefits, ANIL suffers from issues like accuracy variance, slow initial learning, and overfitting, hardening its adaptation and generalization. This work proposes “Look-Ahead ANIL” (LaANIL), an enhancement to ANIL for better learning. LaANIL reorganizes ANIL’s internal architecture, integrating parallel computing techniques (to process multiple training examples simultaneously across computing units) and incorporating Nesterov momentum (which accelerates convergence by adjusting the learning rate based on past gradient information and extracting informative features for look-ahead gradient computation). These additional features make our model more state-of-the-art capable and better edge-compatible and thus improve few-short learning by enabling models to quickly adapt to new information and tasks. LaANIL’s effectiveness is validated on established meta-few-shot learning datasets, including FC100, CIFAR-FS, Mini-ImageNet, CUBirds-200-2011, and Tiered-ImageNet. The proposed model achieved an increased validation accuracy by 7 ± 0.7% and a variance reduction by 44 ± 4% in two-way two-shot classification as well as increased validation by 5 ± 0.4% and a variance reduction by 18 ± 2% in five-way five-shot classification on the FC100 dataset and similarly performed well on other datasets.

With over 1.6 million traffic deaths in 2016, automated vehicles equipped with automated driving systems (ADSs) have the potential to increase traffic safety by assuming human driving tasks within the operational design domain (ODD). However, safety validation is challenging due to the open-context problem. Current strategies, such as pure driving and requirement-based testing, are insufficient. Scenario-based testing offers a solution but necessitates appropriate scenario selection, testing methods, and evaluation criteria. This paper builds upon a method to calculate the covered ODD using tested scenarios generated from logical scenarios, considering parameter discretisation uncertainty. Acceptance criteria for the safety argumentation are proposed based on parameter space coverage and variance introduced via discretisation, thus contributing to quantifying the residual risks of safety validation. The approach is demonstrated through two logical scenarios with probability density functions of the parameters generated using a trajectory dataset. These criteria can serve as risk acceptance criteria, providing comparability and explainable results. By developing a robust scenario-based testing approach, ADS safety can be validated, leading to increased traffic safety and reduced fatalities. Since ADSs incorporate AI models, this proposed validation strategy can be extended to AI systems across multiple domains for the respective assurance argument required for deployment.

Automated Vehicles (AVs) are designed to enhance road safety by utilizing Automated Driving Systems (ADS) that leverage behavioral competencies within the targeted Operational Design Domain (ODD). However, operation within the current ODD always carries a residual risk that must be kept within acceptable limits to ensure safe and robust operation. This paper proposes a system-level residual risk management strategy for ACC/LKA behavioral competencies, which comprises a receive-monitor-transmit concept for hierarchical monitoring functionalities, a system-level residual risk management strategy, and fault injection campaigns to challenge the implemented multi-layer monitoring functionalities. The proposed strategy is implemented ACC/LKA-driven benchmark example, which demonstrates the efficient and effective handling of residual risks at the system level. The study concludes that targeted ODD and/or related behavioral competence reductions are a promising approach to maintaining the residual risk within acceptable limits.

Monitoring the functionality of systems during operation is vital for detecting faults and preventing their consequences.In autonomous driving, monitoring is even more critical because of hardly being able to verify all implemented functionality.Today, systems comprise many interacting components making centralized monitoring less feasible and hard to handle.Hence, we suggest a distributed but connected monitoring system that reflects the system's conceptual structure.In this paper, we outline the foundations of a monitoring system, present some applications and show how we use concepts like the operational design domain and requirements for obtaining the required monitoring knowledge in the application area of autonomous driving.

Automated Vehicles aim to increase road safety as automated driving systems (ADS) take over the human driving task in the operational design domain (ODD), introducing severe challenges for safety validation. Pure driving over many kilometers to gather enough evidence for a safety argument is not feasible. Scenario-based testing is an approach to overcome this, but challenges like parameter discretization still prevail, hindering safety assurance. This work proposes contributions towards a traceable and efficient safety argumentation for ADS built upon ODD coverage. First, ODD coverage is thoroughly quantified across all scenario levels, assuming distribution functions’ availability for the scenario parameters. Secondly, a sampling method for n-dimensional scenario parameter distributions is proposed. The provided algorithms adapt an initial k-means clustering using pre-defined boundary conditions requiring significantly fewer scenarios. Furthermore, a risk metric for urban intersections is presented for scenario evaluation. The risk metric consists of two parts, scene prediction of traffic participants (TPs) and risk assessment. The scene prediction uses a manoeuvre-based motion model with a data-driven approach towards trajectory prediction, increasing the validity. For the risk assessment, a probabilistic risk prediction for the TPs is performed for each scenario scene. The risk metric shows a reasonable tradeoff between sensitivity and specificity, outperforming time-to-collision. These contributions are exemplarily applied at an intersection using a simplified setup for generating TPs and ego vehicle trajectories. The results indicate that an increased safety argumentation is enabled using the proposed methods alongside a coverage process, facilitating further research.

The increasing number of electronic devices in our everyday life demands for convenient input methods. At the same time, for climate protection, low power consumption of the related human machine interfaces (HMI) is a key requirement. This research deals with finding a suitable processing pipeline for a novel, energy-efficient hand gesture recognition (HGR) method based on airborne ultrasound waves. For this purpose, a new dataset was recorded. The data was stored as distance values, raw spectrograms, and pre-processed spectrograms and processed with a variety of classification algorithms. All datasets and classification algorithms turned out to be suitable for successful HGR. Another conclusion is, that preprocessing is not required in case of spectrogram-based classification and basic HGR was possible without machine learning (ML). Considering these outcomes, HGR based on ultrasonic data is promising and worth further investigation.

Accurate and robust object detection is critical for autonomous driving. Image-based detectors face difficulties caused by low visibility in adverse weather conditions. Thus, radar-camera fusion is of particular interest but presents challenges in optimally fusing heterogeneous data sources. To approach this issue, we propose two new radar preprocessing techniques to better align radar and camera data. In addition, we introduce a Multi-Task Cross-Modality Attention-Fusion Network (MCAF-Net) for object detection, which includes two new fusion blocks. These allow for exploiting information from the feature maps more comprehensively. The proposed algorithm jointly detects objects and segments free space, which guides the model to focus on the more relevant part of the scene, namely, the occupied space. Our approach outperforms current state-of-the-art radar-camera fusion-based object detectors in the nuScenes dataset and achieves more robust results in adverse weather conditions and nighttime scenarios.

Virtual testing using simulation will play a significant role in future safety validation procedures for automated driving systems, as it provides the needed scalability for executing a scenario-based assessment approach. This article combines multiple essential aspects that are necessary for the virtual validation of such systems. First, a general framework that contains the vital subsystems needed for virtual validation is introduced. Secondly, the interfaces between the subsystems are explored. Additionally, the concept of model fidelities is presented and extended towards all relevant subsystems. For an automated lane-keeping system with two different definitions of an operational design domain, all relevant subsystems are defined and integrated into an overall simulation framework. The resulting difference between both operational design domains is the occurrence of lateral manoeuvres, leading to greater demands of the fidelity of the vehicle dynamics model. The simulation results support the initial assumption that by extending the operation domain, the requirements for all subsystems are subject to adaption. As an essential aspect of harmonising virtual validation frameworks, the article identifies four separate layers and their corresponding parameters. In particular, the tool-specific co-simulation capability layer is critical, as it enables model exchange through consistently defined interfaces and reduces the integration effort. The introduction of this layered architecture for virtual validation frameworks enables further cross-domain collaboration.

Automated Driving Systems (ADSs) commend a substantial reduction of human-caused road accidents while simultaneously lowering emissions, mitigating congestion, decreasing energy consumption and increasing overall productivity. However, achieving higher SAE levels of driving automation and complying with ISO26262 C and D Automotive Safety Integrity Levels (ASILs) is a multi-disciplinary challenge that requires insights into safety-critical architectures, multi-modal perception and real-time control. This paper presents an assorted effort carried out in the European H2020 ECSEL project—PRYSTINE. In this paper, we (1) investigate Simplex, 1oo2d and hybrid fail-operational computing architectures, (2) devise a multi-modal perception system with fail-safety mechanisms, (3) present a passenger vehicle-based demonstrator for low-speed autonomy and (4) suggest a trust-based fusion approach validated on a heavy-duty truck.