Large language models are prone to misuse and vulnerable to security threats, raising significant safety and security concerns. The European Union's Artificial Intelligence Act seeks to enforce AI robustness in certain contexts, but faces implementation challenges due to the lack of standards, complexity of LLMs and emerging security vulnerabilities. Our research introduces a framework using ontologies, assurance cases, and factsheets to support engineers and stakeholders in understanding and documenting AI system compliance and security regarding adversarial robustness. This approach aims to ensure that LLMs adhere to regulatory standards and are equipped to counter potential threats.
In operational technology (OT) contexts, containerised applications often require elevated privileges to access low-level network interfaces or perform administrative tasks such as application monitoring. These privileges reduce the default isolation provided by containers and introduce significant security risks. Security risk identification for OT container deployments is challenged by hybrid IT/OT architectures, fragmented stakeholder knowledge, and continuous system changes. Existing approaches lack reproducibility, interpretability across contexts, and technical integration with deployment artefacts. We propose a model-based approach, implemented as the Container Security Risk Ontology (CSRO), which integrates five key domains: adversarial behaviour, contextual assumptions, attack scenarios, risk assessment rules, and container security artefacts. Our evaluation of CSRO in a case study demonstrates that the end-to-end formalisation of risk calculation, from artefact to risk level, enables automated and reproducible risk identification. While CSRO currently focuses on technical, container-level treatment measures, its modular and flexible design provides a solid foundation for extending the approach to host-level and organisational risk factors.
Government as a Platform (GaaP) represents the notion that governments significantly benefit from the adoption of platform concepts from the private sector. While several governments are already implementing this notion, there is an insufficient understanding of what actually constitutes a successful GaaP approach. We apply a design science research strategy and, based on interviews with 26 experts and a survey, iteratively design and evaluate 10 guiding principles for GaaP. The principles show how technological and governance aspects complement each other and which platform concepts constitute GaaP. Our research has implications for scholars towards a detailed conceptualization of GaaP, while practitioners can use the principles as guidelines for implementation.
Software defines digital infrastructures in the manufacturing industry, connecting services and computation resources to machines and devices. These infrastructures aim at increased flexibility, scalability, and a wider application portfolio for automated manufacturing processes. At the same time, the complexity of securing software increases dramatically. Threats to confidentiality, integrity, and availability of software can result in critical losses for automated industrial production and impact manufacturing companies. In order to map existing and emerging security challenges, we present the results of a hermeneutic literature review structured along abstraction levels and vertical integration of software. Based on this structure, we derive implications for academia and practice focused on system integrators, developers, and security auditors of digital infrastructures. Thereby, we discuss courses of action mapped to software security black boxes, infrastructure heterogeneity, and the adaptation of security for operational usage.
. Government as a Platform (GaaP) is a promising approach to the digital transformation of the public sector. In practice, GaaP is realized by platform-oriented infrastructures. However, despite successful examples, the transformation toward platform-oriented infrastructures remains challenging. A potential remedy is the analysis of existing public infrastructure regarding its platform orientation. Such an analysis can identify the gaps to an ideal platform-oriented infrastructure and, thus, support the transformation toward it. We follow the design science research methodology to develop a four-dimensional analysis method. We do so in three iterations, and, after each iteration, evaluate the method by its application to infrastructures in practice. With regard to theory, our results suggest extending GaaP conceptualizations with a specific emphasis on platform principles. With regard to practice, we contribute an analysis method that creates proposals for the improvement of infrastructures and, thus, supports the transformation toward GaaP.
Assurance cases (ACs) are a common artifact for building and maintaining confidence in system properties such as safety or robustness. Constructing an AC can be challenging, although existing tools provide support in static, document-centric applications and methods for dynamic contexts (e.g., autonomous driving) are emerging. Unfortunately, managing ACs remains a challenge, since maintaining the embedded knowledge in the face of changes requires substantial effort, in the process deterring developers - or worse, producing poorly managed cases that instill false confidence. To address this, we present OntoGSN: an ontology and supporting middleware for managing ACs in the Goal Structuring Notation (GSN) standard. OntoGSN offers a knowledge representation and a queryable graph that can be automatically populated, evaluated, and updated. Our contributions include: a 1:1 formalization of the GSN Community Standard v3 in an OWL ontology with SWRL rules; a helper ontology and parser for integration with a widely used AC tool; a repository and documentation of design decisions for OntoGSN maintenance; a SPARQL query library with automation patterns; and a prototypical interface. The ontology strictly adheres to the standard's text and has been evaluated according to FAIR principles, the OOPS framework, competency questions, and community feedback. The development of other middleware elements is guided by the community needs and subject to ongoing evaluations. To demonstrate the utility of our contributions, we illustrate dynamic AC management in an example involving assurance of adversarial robustness in large language models.
The EU AI Act (EUAIA) introduces requirements for AI systems which intersect with the processes required to establish adversarial robustness. However, given the ambiguous language of regulation and the dynamicity of adversarial attacks, developers of systems with highly complex models such as LLMs may find their effort to be duplicated without the assurance of having achieved either compliance or robustness. This paper presents a functional architecture that focuses on bridging the two properties, by introducing components with clear reference to their source. Taking the detection layer recommended by the literature, and the reporting layer required by the law, we aim to support developers and auditors with a reasoning layer based on knowledge augmentation (rules, assurance cases, contextual mappings). Our findings demonstrate a novel direction for ensuring LLMs deployed in the EU are both compliant and adversarially robust, which underpin trustworthiness.
Industrial Internet of Things (IIoT) platforms connect services and computation resources to industrial devices. They increase flexibility, scalability, and provide a wider application portfolio for automated production. However, in a growing ecosystem of software suppliers, manufacturing companies are concerned whether the security of third-party software meets the requirements of the industry. In this paper, we analyse IIoT reference architecture components and corresponding stakeholders to draw implications on security. In particular, we identify four security challenge areas of IIoT platforms, show relations to existing research, and highlight directions for future work.
Despite the impressive adaptability of large language models (LLMs), challenges remain in ensuring their security, transparency, and interpretability. Given their susceptibility to adversarial attacks, LLMs need to be defended with an evolving combination of adversarial training and guardrails. However, managing the implicit and heterogeneous knowledge for continuously assuring robustness is difficult. We introduce a novel approach for assurance of the adversarial robustness of LLMs based on formal argumentation. Using ontologies for formalization, we structure state-of-the-art attacks and defenses, facilitating the creation of a human-readable assurance case, and a machine-readable representation. We demonstrate its application with examples in English language and code translation tasks, and provide implications for theory and practice, by targeting engineers, data scientists, users, and auditors.
Establishing and assuring compliance of information systems is a difficult task with potentially critical impact to the security of those same systems. Ambiguously worded laws such as the Digital Operational Resilience Act make it difficult for organizations to determine which actions to undertake in pursuit of compliance. This ambiguity prompts auditors, compliance officers and other involved roles to interpret the meaning and implement measures according to best judgment, resulting in an intricate back-and-forth process with remaining uncertainties. In a qualitative case study involving a multinational financial corporation and its information systems, we explore the needs of stakeholders that emerge from the interpretations and uncertainties in the process. We model the complex interconnections in a figure from a deeper look in the subcase on establishing and assuring compliance of identity and access management (IAM) procedures. Finally, we discuss potential avenues for resolving these problems.
Assuring regulatory compliance of information systems (IS), as a bundle of software systems and business processes, is an important, but costly and continuous effort. Laws formulate demands for quality properties in ambiguous language, requiring substantial interpretation. Industry standards provide support, but remain generic and applicable to heterogeneous company IS contexts. Before compliance measures can be implemented in software assets and processes, a specific interpretation based on the context of each company is a prerequisite. Compliance experts such as auditors support this process by accounting for the perspectives of company stakeholders. Ultimately, however, the complexity of the required knowledge, legal and technical facets prevents organizations from continuously establishing situational awareness or guarantees, and answering the question: is the company currently compliant? We illustrate the complexity of assuring compliance in a qualitative case study with a European, software-driven corporation in the financial industry. Through modeling of an example of annual audits and analyzing literature, we describe the perspectives of the involved stakeholders with their roles, knowledge needs and facets. We observe six challenges: (1) large number of items and links; (2) unclear and implicit links; (3) siloing of knowledge; (4) multiple sources of truth; (5) high costs of learning from audits; and (6) uncertain results of traditional auditing. We discuss the implications of these observed challenges, and briefly explore potential avenues for resolution.
This paper presents an approach to developing assurance cases for adversarial robustness and regulatory compliance in large language models (LLMs). Focusing on both natural and code language tasks, we explore the vulnerabilities these models face, including adversarial attacks based on jailbreaking, heuristics, and randomization. We propose a layered framework incorporating guardrails at various stages of LLM deployment, aimed at mitigating these attacks and ensuring compliance with the EU AI Act. Our approach includes a meta-layer for dynamic risk management and reasoning, crucial for addressing the evolving nature of LLM vulnerabilities. We illustrate our method with two exemplary assurance cases, highlighting how different contexts demand tailored strategies to ensure robust and compliant AI systems.
Industrial Internet of Things (IIoT) platforms are characterised by a heterogeneity of applications that increases the flexibility and efficiency of automated manufacturing. At the same time, manufacturers are concerned whether untrusted third-party applications can be aligned with the domain’s requirements for security. Assurance activities, aiming to guarantee the secure development and operation of applications, are hampered by risk management challenges and the heterogeneity of required knowledge. Stakeholders such as Software Developers and System Integrators struggle to break down these challenges, e.g. how to mitigate threats to Industrial Control Systems (ICS) and derive actions for assurance. Additionally, they require knowledge to manage risks from heterogeneous facets such as application deployment, device configuration, or threat and risk assessment. In this paper, we propose an assurance engine that allows stakeholders to break down risk management challenges into dynamic assurance cases and to augment the latter with knowledge from corresponding facets. By leveraging knowledge representation, the assurance engine enables the comparison of risk management approaches for different stakeholders. Furthermore, we utilise formal semantics and logical deduction for reasoning to lay the grounds for the automated assessment of complex assurance cases on heterogeneous IIoT platforms.
In its 14 years, distributed ledger technology has attracted increasing attention, investments, enthusiasm, and user base. However, ongoing doubts about its usefulness and recent losses of trust in prominent cryptocurrencies have fueled deeply skeptical assessments. Multiple groups attempted to disentangle the technology from the associated hype and controversy by building workflows for rapid prototyping and informed decision-making, but their mostly isolated work leaves users only with fewer unclarities. To bridge the gaps between these contributions, we develop a holistic analytical framework and open-source web tool for making evidence-based decisions. Consisting of three stages - evaluation, elicitation, and design - the framework relies on input from the users' domain knowledge, maps their choices, and provides an output of needed technology bundles. We apply it to an example clinical use case to clarify the directions of our contribution charts for prototyping, hopefully driving the conversation towards ways to enhance further tools and approaches.
Government as a Platform (GaaP) promises better and more efficient public services. More and more countries are applying the approach to eGovernment development. However, there is scant empirical evidence on how to properly implement GaaP in practice. In particular, most of the literature focusses on the adoption in individual countries. We address this gap by investigating and systematically comparing three countries with successful GaaP implementations. By means of expert interviews and analysis of public documents we are able to extract four commonalities and three differences. We discuss our results as lessons learnt. We further contribute to theory and practice by enhancing the knowledge on GaaP approaches, providing a first basis for guidelines toward GaaP.
Organizations in highly regulated domains often struggle to build well-performing machine learning (ML) models due to restrictions from data protection regulation. Federated learning (FL) has recently been introduced as a potential remedy, whereby organizations share local models while keeping data on premise. Still, regulatory compliance remains challenging in FL settings: training data needs to be shared to some extent, and models can be reverse engineered or misused towards violation of data privacy by each participating organization. Guided by design science methodology, we introduce four interaction patterns that allow for compliance-by-design and trust-context-sensitive analysis of an FL system by combining different approaches to privacy preservation. We match the patterns to privacy principles and exemplify how verifiable claims about compliance at design-and operation-time FL can be generated to make all participating organizations accountable.
Sustainability-aware grocery shopping can be challenging for consumers since product information becomes more and more extensive. Especially the variety and trustworthiness of labels makes it difficult to find the most sustainable products, in particular with respect to the heterogenic and varying level of detail claimed by producers based on these labels. In order to decrease information overload and increase consumer protection, we develop a government-driven platform that fuses heterogeneous food data and provides chatbot-like AI services to communicate a sustainability score. We demonstrate the use and potential of the government-governed platform with two prototypical interfaces. Furthermore, we present first insights from the cooperation with start-ups who integrate the data platform into their products.
Federated Learning (FL) is a novel paradigm for the shared training of models based on decentralized and private data. With respect to ethical guidelines, FL is promising regarding privacy, but needs to excel vis-\`a-vis transparency and trustworthiness. In particular, FL has to address the accountability of the parties involved and their adherence to rules, law and principles. We introduce AF^2 Framework, where we instrument FL with accountability by fusing verifiable claims with tamper-evident facts, into reproducible arguments. We build on AI FactSheets for instilling transparency and trustworthiness into the AI lifecycle and expand it to incorporate dynamic and nested facts, as well as complex model compositions in FL. Based on our approach, an auditor can validate, reproduce and certify a FL process. This can be directly applied in practice to address the challenges of AI engineering and ethics.