AI-enabled functions are increasingly considered for aerospace and other safety-critical systems, where predictable execution is as important as acceleration. Standard GPU programming models such as CUDA do not inherently provide the timing guarantees required for certification, since runtime jitter and limited worst-case predictability remain problematic. Vulkan SC has been proposed as a more deterministic alternative for safety-critical GPU execution. This work compares CUDA and Vulkan SC on the NVIDIA Jetson Orin Nano SoC platform with a focus on timing determinism. We apply fixed clocking, warm-up runs, and process-level optimizations. We then analyze timing contributions from memory management, data movement, and kernel execution. The evaluation covers neural-network-related synthetic primitives, composite SAXPY-ReLU chains, and a fused neural-network workload. To relate the synthetic primitives to the NN workload, we compare an operation-count-equivalent unfused primitive baseline with the measured fused implementation across input sizes. The results show that Vulkan SC provides more stable kernel execution and consistently low kernel-side jitter. The growth analysis shows that CUDA kernel time is nearly input-size independent for small and medium input sizes, while Vulkan SC shows less uniform growth. For both APIs, the unfused baseline remains above the fused NN kernel time, while the relative gap decreases with increasing input size. H2D/D2H transfer timing scales almost linearly with transferred data size for larger transfers, but remains a major source of end-to-end variability, especially for Vulkan SC. Overall, Vulkan SC improves kernel-level timing determinism, while system-level predictability remains constrained by transfer overhead and shared-memory effects.
Open-source operating systems, e.g., FreeRTOS and Zephyr, are developed by distributed communities that typically follow non-standardized, ad-hoc software development practices. This makes their adoption in safety-critical contexts particularly challenging, especially when targeting the highest safety assurance levels. This work presents a software architecture designed to remove the operating system from the safety rationale. This is accomplished by leveraging the stage-2 memory protection unit (MPU) available in the hypervisor mode of modern real-time processors. In contrast to existing (hypervisor-based) approaches, the proposed concept provides finer granularity by enabling the protection of individual applications rather than entire virtual machines. Our evaluation on the latest generation of NXP embedded SoCs (S32N family) shows that the combined overhead of control-flow interception and the reprogramming of the stage-2 MPU for each transition into and out of a safety-critical application is only about 0.5 s, which is comparable to typical interrupt path latencies. This indicates that the proposed scheme remains suitable even for low-latency, hard real-time applications.
Machine learning models for natural language processing have achieved remarkable performance across many domains, yet their adoption in safety engineering remains limited. A key barrier are hazardous failure conditions - such as hallucinations - which arise from an insufficient functionality of a foundation model system and must be controlled in safety-critical contexts. This article examines such behavior through a safety lens and traces root causes for obscure hazardous failure conditions in generic foundation model setups. From this analysis, we distill five broadly applicable concepts that provide a structured way to counter these challenges (narrow domain, setup diligence, uncertainty estimation and interpretability, iterative approaches, hybrid approaches). Together, they offer a principled starting point for deriving concrete design and assessment practices for safety engineers working with language-model-based systems. We illustrate our approach through three representative use cases that demonstrate how these five countermeasures can be applied in real-world safety engineering scenarios.
Integrating Machine Learning models within certified safety critical avionics systems faces several challenges among which we focus on the difficulty to demonstrate that a trained Machine Learning Model meets its requirements when deployed on the target system. Indeed, a semantic gap exists between training framework models and deployed inference models on target hardware. Certification standards require the rigorous implementation of ML model descriptions with design assurance, as well as a demonstration that properties verified during training (such as accuracy and robustness) are preserved on the target. To that end, we propose the Target Aware Training (TAT) approach, that ensures semantic preservation by construction. Practically, the training framework’s native inference engine is substituted with a compiled shared library derived from design-assured code for the target hardware. Doing so, we achieve bit-accurate replication and eliminate the need for error-prone transformations to and from intermediate representations like ONNX. The approach also allows the specification, control, and formal verification of activation and accumulator bounds. We validate our approach on the ResNet18 architectures, guaranteeing the absence of numerical overflows even for unseen data through a generative verification method. This dual contribution provides a robust path toward meeting DO-178C design assurance objectives for airborne ML applications.
Reliable Positioning, Navigation, and Timing (PNT) is a fundamental requirement for the safe operation of autonomous vehicles. While traditional GNSS-based PNT is typically very accurate in open sky conditions, its performance can degrade significantly in environments where satellite visibility is limited or signals are obstructed. To ensure continuity, it is beneficial to complement GNSS with other methods, such as AI-enhanced Visual SLAM, which remain robust under challenging environmental conditions. However, AI-based components can produce untrustworthy results and necessitate a safety cage to monitor and filter their output. This experience report presents the VAIPOSA architecture, which introduces a safety cage to perform runtime verification and fusion of hybrid GNSS and AI-based PNT sources. The safety cage monitors a variety of parameters, including GNSS signal characteristics, AI-based precision metrics, image quality, and vehicle state information, to determine the trustworthiness of each source. We describe the integration of these monitors within a supervisor that fuses data into a reliable pose estimate and discuss its validation in the CARLA simulator using automated test case generation and fault injection scenarios.
The increasing complexity of safety-critical systems demands rigorous hazard analysis, especially for capturing unsafe interactions between subsystems. System-Theoretic Process Analysis (STPA) is an effective way to identify unsafe system interactions, associated hazards, and loss scenarios, and then derive safety requirements. Becoming proficient with STPA requires significant understanding of control actions, which is a challenging concept to novices. We leverage repetitions in the derivation of unsafe control actions and loss scenarios of STPA to propose modifications that help more analysts get the most out of STPA. These modifications result in a more systematic approach to conducting STPA. They also help lay the groundwork for the risk assessment that many companies must perform. We demonstrate the application of the modified STPA in an industrial application – a remote and virtual tower system for air traffic services.
As the number of automotive safety standards increases, including ISO 26262, ISO 21448 (Safety of the Intended Functionality, SOTIF), and UL 4600, practitioners need evidence on how these standards are being adopted in industry. This practical experience report presents a questionnaire-based study conducted within JASPAR, a Japanese automotive consortium, through its Functional Safety Working Group, with 30 responses from 28 of 40 companies (70
Runtime verification checks, during execution, whether a system satisfies a given property. When a violation is detected, identifying the underlying causes is essential for fault recovery. However, existing approaches to model-based diagnosis do not fully address this problem. Model-Based Diagnosis (MBD) explains deviations between observed and nominal behaviors, but does not target property violations specifically. Model-Based Safety Analysis (MBSA), on the other hand, explains possible causes of a top-level event, but does not exploit runtime observations. In this work, we address the Property Violation Diagnosis (PVD) problem, which lies at the intersection of runtime verification and model-based diagnosis: given a system description, a monitored property, and a set of observations that indicate its violation, we aim to compute the set of faults that are relevant to explain the violation. We provide a formal characterization of PVD in propositional logic and propose symbolic algorithms for generating the relevant diagnoses. Our experimental evaluation demonstrates the applicability of the proposed approach.
With shrinking structure sizes, radiation-induced faults in microelectronic components are a concern for the design of high-reliability systems. As radiation testing is expensive, non-reproducible, and only possible after a chip has been fabricated, tools for analyzing the effect of faults on ASIC designs during the design phase are needed. FPGA-based fault-injection analysis can provide a speed-up compared to simulation, enabling higher fault-space coverage. This work presents OpenFI4ASIC, a modular open-source framework for performing fault-injection campaigns on layout netlists of ASIC designs. Faults are injected by replacing standard-cell instances with instrumented instances connected via a scan chain. This instrumentation is performed automatically by the OpenFI4ASIC framework, which also provides the hardware interfaces and software drivers to facilitate fault-emulation campaigns on an SoC FPGA. As a case study, a configurable open-source RISC-V core is evaluated for five implementations, from single-cycle to five-stage pipelined, using three standard benchmarks (ISQRT, BUBBLESORT, and CRC32). The cores are synthesized for 45nm NanGate technology, and the resulting netlists are used as input for OpenFI4ASIC and emulated on a Zynq-7000 SoC FPGA. The generated FPGA projects enable fault-injection campaigns at up to 6878.76 fault-injection runs per second, allowing full coverage of the single-bit fault within minutes. The results highlight the importance of considering the architecture implementation, showing differences of up to 30.3
In many safety-critical applications, binary classifiers detect undesirable operational states. Optimal adjudication schemes may provide an affordable means of constructing Bayes-optimal hybrid classifiers (i.e., a classifier configuration with the lowest expected cost of classification errors) by combining the outputs from multiple classifiers. However, statistical uncertainty poses significant challenges when applying such schemes, so that their use cannot be guaranteed to produce truly optimal configurations. We present statistical methods for estimating the extent to which various adjudication schemes, including optimal adjudication, fail to produce optimal classifier configurations. Applying these methods to two critical contexts—pneumonia diagnosis and concrete-defect detection—classical statistical bounds reveal how training-sample uncertainty, validation-sample uncertainty, and the relative cost of classification errors jointly constrain both the efficacy of adjudication schemes and the confidence that can be placed in the configurations they produce. So-called “smoothed” hybrid classifiers consistently give the lowest risk with the tightest optimality confidence bounds, particularly when the relative cost of classification errors is very low/high.
Security in industrial automation systems becomes more complicated as industry moves toward modern and flexible solutions such as modular automation and increased vendor diversity. Attack surfaces grow as more network interfaces are added and more devices become exposed to the internet and cloud. The rapid introduction and replacement of devices make it difficult to track vulnerabilities and ensure nothing malicious enters the system. Hence, trusting a device prior to its onboarding becomes critical for ensuring the overall security and safety of the system. This paper proposes a remote attestation mechanism for the onboarding process which follows the Remote ATtestation ProcedureS proposals. In order to provide evidence for the applicability of the proposed process, we provide a reference implementation. Moreover, we provide a security analysis of the proposed solution in the form of a threat model.
We present a new method for assessing probabilistic confidence in assurance case arguments that is simple, systematic and sound. It exploits the ways claims are decomposed in a structured argument and provides different approaches according to the different degrees of (in)dependence and diversity among subclaims, and how they eliminate the concerns that undermine confidence in their parent claims. The method uses only elementary probabilistic constructions that are well-known in other contexts (e.g., Fréchet bounds) but we interpret and apply them in a manner that is specifically focused on assurance arguments and requires no background in probabilistic analysis. We show that the method is not susceptible to counterexamples that Graydon and Holloway exhibit for other approaches to quantifying confidence [12] and we recommend it as an additional tool and point of view in assessing Assurance 2.0 arguments that can contribute to dialectical examination and evaluation of cost/confidence tradeoffs for different risk levels and the overall balance of confidence across a structured argument.
The increasing interconnection of Industrial Computer Systems heightens the risk of attacks on Industrial Control Systems (ICS). Remote attackers can interfere with the control room’s ability to monitor and control a physical process, impairing operational safety. To assess or trace the impact of such attacks, digital forensics are essential, where data is identified and collected to generate evidence. Due to differences between office IT and operational technology (OT) used in ICS, established IT forensics cannot be applied directly, while existing approaches only cover the partial acquisition of control room related data. However, to integrate relevant data into acquisition or forensic methods, it must be identified where control room related data resides and which software processes and connections are of relevance. As ICS software, protocols, and data are often proprietary and vendor-specific, a vendor-independent identification approach is required. We propose a novel runtime-based reverse engineering procedure for identifying control room related data, paths, and software processes in Windows-based ICS environments. This enables one to determine which processes and connections are of forensic or safety relevance and need to be covered regarding acquisition or security-monitoring for control room operations. To establish our approach, we analyze how control and monitoring data is exchanged internally on an ABB 800xA Distributed Control System (DCS) and validate the derived approach by applying it to a Siemens Process Control System (PCS7) DCS/SCADA.
Security testing frameworks provide large collections of executable test cases, but running all of them is often impractical. Existing test case prioritization (TCP) methods typically rely on historical execution outcomes, coverage information, or project-specific internal signals, which are often unavailable or insufficient for security testing. We propose a threat-informed and uncertainty-aware TCP framework that instead prioritizes tests using external threat intelligence and features extracted directly from test descriptions. Using Atomic Red Team as a case study, we enrich each test with global vulnerability severity evidence through ATT CK–CVE mappings and combine this with local operational features derived from the test definitions. Because many tests lack linked vulnerability evidence, we introduce a learned global severity imputer trained on evidence-backed techniques and used to estimate severity for unmapped ones. Regression models, including Random Forest, ExtraTrees, and quantile gradient boosting, are then trained to predict priority scores and estimate uncertainty. To make uncertainty actionable, we learn symbolic rules with a Tsetlin Machine that separates high- and low-uncertainty cases and supports human review. Results show that uncertainty profiles differ across models and that high uncertainty is frequently associated with sparse or inconsistent vulnerability evidence, including cases where CVEs exist but supporting fields are incomplete. These findings support a practical human-in-the-loop workflow in which uncertain high-priority tests are flagged for expert review and threat-intelligence enrichment. Overall, the framework provides a practical alternative to history-based TCP by enabling learned, threat-informed prioritization under incomplete evidence. While the current work focuses on learning a prioritization policy from threat intelligence and test-case characteristics, the resulting models provide a foundation for future system-aware prioritization using deployment-specific contextual information.
Subsea data cables carry the vast majority of intercontinental data traffic, yet existing regional risk assessments have relied on simulated inputs or narrow-area studies. SubSeaCure provides a grid-based methodology that uses observational data to quantify and visualize external risks to subsea data cables across Europe’s adjacent seas. Six risk factors (fishing, anchorage, dredging, seismic activity, volcanic activity, and seabed ruggedness) were mapped to a raster grid and normalized to a common scale. Using a weighted-sum model derived from the Analytic Hierarchy Process (AHP), the factors were aggregated into a combined risk score dominated by fishing and anchorage as the most prevalent causes of cable incidents. The resulting risk map revealed pronounced spatial heterogeneity. The most extensive high-risk belt spans the North Sea and English Channel, with particularly elevated risk around Denmark, Germany, and the Netherlands. Further high-risk concentrations were also observed in the northeastern Mediterranean. Offshore, deeper Atlantic areas, and parts of the Black Sea exhibited comparatively low risk levels. These results provide contextual input for public and private investment considerations, route engineering, targeted protection and burial, surveillance prioritization, or plausibility checks during incident assessment. The SubSeaCure methodology is scalable and provides a template for finer-grained risk mapping of seabed infrastructure networks in European seas and beyond.
Fault tolerance in Internet of Medical Things (IoMT) systems is challenging due to heterogeneous architectures, stringent safety requirements, and concurrent exposure to cyberattacks and performance degradation. This paper proposes a two-layer adaptive fault tolerance framework integrating lightweight intrusion detection with domain-aware anomaly quantification. The first layer employs a machine learning-based intrusion detection system to produce a binary safety score. The second layer uses a domain-weighted autoencoder to compute a continuous Health Index (HI) from Quality-of-Service metrics and the safety score. The principal contribution is the systematic demonstration that feature weighting within the reconstruction loss plays a central role in safety-critical detection, improving Critical Recall from 7.3 ϵ -greedy Q-learning agent, and a Teacher–Student knowledge distillation pipeline compresses cloud-level intelligence for edge deployment, achieving a 34.6× reduction in computational cost while preserving Critical Recall. Experiments on the WUSTL-EHMS-2020 dataset validate the framework against ablation baselines and a structurally comparable state-of-the-art method.
Safety cases rely on expert experience often formalized in natural language reports about accidents, incidents, and defects. Discovering relevant information in this growing body of knowledge is complex. We present a traceable, closed-corpus hybrid retrieval-augmented generation (RAG) pipeline for automatic defeater generation from autonomous driving accident reports. It combines sparse term frequency - inverse document frequency (TF-IDF) and dense semantic retrieval to extract relevant counter evidence from incidents issued by the NTSB, NHTSA, and Cruise. The pipeline parses the excerpts to a large language model (LLM) for generating defeaters. We evaluated our approach using the Open Autonomous Safety Case Framework’s (OASCF) high-level safety argument. Three domain experts assessed independently the generated defeaters based on their logical relevance. Two experts independently evaluated the same subset of about 167 generated defeaters on a scale of 0–2. The inter-rater agreement was measured using Cohen’s kappa (k = –0.01), indicating poor agreement. A third domain expert provided a qualitative in-depth analysis of the defeaters and an overall discussion, confirming the conceptual soundness of the approach. The ratio of strongly identified defeaters to weak and non-defeaters was about 1:3. The method is proposed for safety engineers examining the argument coverage in the preliminary safety case stage, reducing manual corpus review efforts while maintaining full traceability to the extracted source evidence.
Developing safety-critical systems require compliance with multiple standards which impose numerous requirements on development and assurance processes. However, understanding, filtering, and operationalizing relevant requirements demands substantial effort and expertise. The shortage of experienced engineers and the growing number of applicable standards and regulations call for new methods to manage this complexity and improve efficiency. This study investigates how generative AI can support engineers in efficiently extracting concrete requirements from a set of standards. We developed a generative AI-based tool that automatically analyzes standards and proposes a list of requirements based on predefined relevance criteria. The tool is embedded into a human-in-the-loop workflow in which domain experts verify and validate the extracted requirements to ensure high quality and maintain traceability to the original standards. We evaluated our approach to three safety standard documents for automated driving systems in the context of verification and validation. AI-generated requirements were first reviewed by a novice safety engineer, and we measured both the tool’s processing time and the engineer’s verification time. We compared this AI-supported workflow to fully manual analysis in an industry project by engineers with three to eight years of experience where 130 out of 314 clauses are declared as relevant. We observed an 84
The increasing number of test cases and limited time frames often prevent the execution of complete regression test suites following software changes. This paper addresses this challenge in an industrial software testing context, focusing on daily system tests at Siemens Healthineers. We propose a machine learning-based approach for test case prioritisation that ranks test cases according to their relevance to recent source code changes. Two models, a neural network and a random forest, are developed and evaluated using a dataset comprising over 10,000 test case executions. The experimental results show that both models outperform traditional prioritisation techniques, achieving up to a 50
The EU AI Act mandates regulatory sandboxes in each member state to support AI system development under regulatory supervision. However, concrete guidance on their design remains limited. This paper presents lessons from KIRR Real, a sandbox initiative that conducted Legal Quick Checks for 15 AI systems across manufacturing, automation, and human resources. The requirements assessed were based on the AI Act and, where applicable, the Machinery Regulation. Across 15 organizations, we identify recurring challenges in risk classification, the interpretation of unclear concepts such as “self-evolving behavior” under the Machinery Regulation, technical implementation, documentation, and resource allocation, indicating that additional guidance is needed. Current guidance inadequately addresses common practices like fine-tuning pre-trained models, and established AI development practices prove insufficient for AI Act compliance. We derive recommendations for future sandboxes emphasizing early-stage engagement, low-threshold access, and integrated legal-technical expertise, contributing to safe and compliant AI deployment.
