Machine learning (ML)-based systems are becoming increasingly ubiquitous even in safety critical environments. The strength of ML systems, to solve complex problems with a stochastic model, leads to challenges in the testing domain. This motivates us to introduce a rigorous testing method for ML-models and their application environment akin to classical software testing, which is independent of the training process and considers the probabilistic nature of ML. The approach is based on the concept of the Probabilistically Extended ONtology (PEON). In brief, PEON is a an ontology modeling the designated Operational Design Domain (ODD), which is extended by assigning probability distributions to classes and their individual attributes, as well as probabilistic dependencies between these attributes. The relevant statistical key figures like accuracy depend not only on the ML-based model but also strongly on the statistics of the test data set, which we refer to by quality assurance (QA) data set, to emphasize its independence from the test data set in the training process. This implies that we have to consider the statistical properties of the QA data in order to evaluate an ML-based system. In this paper we present first experimental results comparing established test selection methods e.g. N-wise, with a new approach the PEON. Our findings strongly suggest, that the underlying statistical properties of the QA data significantly influence the test results of ML-based systems. In this respect, careful attention must be paid to the statistical independence and balance of the QA data. The PEON provides a good basis for the composition of QA data sets, which are not only independent of the development process but also statistically representative and balanced with respect to the modeled ODD.

This article proposes a test procedure that can be used to test ML models and ML-based systems independently of the actual training process. In this way, the typical quality statements such as accuracy and precision of these models and system can be verified independently, taking into account their black box character and the immanent stochastic properties of ML models and their training data. The article presents first results from a set of test experiments and suggest extensions to existing test methods reflecting the stochastic nature of ML models and ML-based systems.

Out-of-distribution detection ensures trustworthiness in machine learning systems by detecting anomalous data points and adjusting confidence in predictions accordingly. However, another key use-case of out-of-distribution detection is the assessment of data quality with respect to a desired distribution or semantic range of data. This work proposes a simple but powerful approach that allows for cleaning of image data based on descriptively defining desired data as well as undesired data. Notably, this method does not require the training of a machine learning model. In addition, this work presents a new image dataset suited for evaluating data cleaning tasks in a way that has practical relevance, and demonstrates satisfactory experimental results.

Typically, machine learning techniques are used to realise autonomous driving. Be it as part of environment recognition or ultimately when making driving decisions. Machine learning generally involves the use of stochastic methods to provide statistical inference. Failures and wrong decisions are unavoidable due to the statistical nature of machine learning and are often directly related to root causes that cannot be easily eliminated. The quality of these systems is normally indicated by statistical indicators such as accuracy and precision. Providing evidence that accuracy and precision of these systems are sufficient to guarantee a safe operation is key for the acceptance of autonomous driving. Usually, tests and simulations are extensively used to provide this kind of evidence. However, the basis of all descriptive statistics is a random selection from a probability space. A major challenge in testing or constructing the training and test data set is that this probability space is usually not well defined. To systematically address this shortcoming, ontologies have been and are being developed to capture the various concepts and properties of the operational design domain. They serve as a basis for the specification of appropriate tests in different approaches. However, in order to make statistical statements about the system, information about the realistic frequency of the inferred test cases is still missing. Related to this problem is the proof of completeness and balance of the training data. While an ontology may be able to check the completeness of the training data, it lacks any information to prove its representativeness. In this article, we propose the extension of ontologies to include probabilistic information. This allows to evaluate the completeness and balance of training sets. Moreover, it serves as a basis for a random sampling of test cases, which allows mathematically sound statistical proofs of the quality of the ML system. We demonstrate our approach by extending published ontologies that capture typical scenarios of autonomous driving systems with probabilistic information.

Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples. Further, these adversarial examples are found to be transferable from the source network in which they are crafted to a black-box target network. As the trend of using deep learning on embedded devices grows, it becomes relevant to study the transferability properties of adversarial examples among compressed networks. In this paper, we consider quantization as a network compression technique and evaluate the performance of transfer-based attacks when the source and target networks are quantized at different bitwidths. We explore how algorithm specific properties affect transferability by considering various adversarial example generation algorithms. Furthermore, we examine transferability in a more realistic scenario where the source and target networks may differ in bitwidth and other model-related properties like capacity and architecture. We find that although quantization reduces transferability, certain attack types demonstrate an ability to enhance it. Additionally, the average transferability of adversarial examples among quantized versions of a network can be used to estimate the transferability to quantized target networks with varying capacity and architecture.

Machine learning systems have gained widespread adoption across various industries. This includes highly regulated ones that need to match certain quality requirements based on a given risk exposure. The MLOps paradigm, following a similar approach to DevOps, promises major improvements in quality and speed, with a focus on deploying ML models at a fast pace with high quality on an automated basis. However, traditional point-in-time certifications with manual audits are inadequate for MLOps setups due to frequent changes to the ML system. To overcome this challenge, we propose Continuous Audit-Based Certification (CABC), which uses automated audits to issue or revoke certificates based on an automated assessment of artifacts from the MLOps lifecycle. Our approach utilizes artifacts from the MLOps lifecycle for quality measurements based on standards such as ISO 25012. We propose a risk-based measurement selection, an audit API for standardized retrieval of data for measurement, a tamper-proof data collection process, and an architecture for separation of duties in the certification process. CABC aims to improve efficiency, enhance trust in the ML system, and support highly regulated industries in achieving their quality goals.

AI development and test is a data driven endeavor. To date it is magnitudes more laborious to collect and annotate training and test data, than to provide a problem matching architecture and train it. In the KI-LOK project, a case study seeks to validate an object recognition system to prevent potentially fatal behavior of autonomous train operations. To accommodate for the vast amount of possible scenarios a train could encounter during operation, we propose a tool chain to automatically generate labeled synthetic images and videos. We start from an ontology of elements such as: Tracks, houses, vehicles or signals, these elements are then sampled and modeled in 3D to represent a scenario. Since the objects and locations of the elements in a scenario are known, no manual annotation or labeling of the data is required. By sampling from an ontology it will be possible to build comprehensive and balanced datasets of scenarios to train and test AI, while adding the benefit of corner case generation by reducing to certain elements in the ontology. This article reports on the current status of the project and the goals it tries to achieve.

Smart software solutions, i.e., software that includes artificial intelligence (AI) and machine learning (ML), have shown a great potential to automate processes that were previously not accessible to automation. These areas include predictive maintenance, the creation of clinical diagnoses, recommendations systems, speech, image and scenario recognition, automated driving etc. Since AI and ML differ from classical software development regarding fundamental activities and processes, it is currently unclear how AI and ML can be integrated into existing industrial-grade software development processes. Addressing the industrialization of ML development and operations, the IML4E project will directly address the specifics of AI and ML by providing interoperability, automation and reusability in the data and the training pipeline. Moreover, IML4E enables continuous quality assurance and supervision for different types of machine learning (supervised learning, unsupervised learning, etc.) throughout the whole life cycle of a smart software solution. In this project presentation, we will focus in particular on the quality assurance and testing research planned in the project.

This article provides a taxonomy for risk-based testing that serves as a tool to define, tailor, or assess such approaches. In this setting, the taxonomy is used to systematically identify deviations between the requirements from public standards and the individual testing approaches.

In order to optimize the usage of testing efforts and to assess risks of software-based systems, risk-based testing uses risk (re-)assessments to steer all phases in a test process. Several risk-based testing approaches have been proposed in academia and/or applied in industry, so that the determination of principal concepts and methods in risk-based testing is needed to enable a comparison of the weaknesses and strengths of different risk-based testing approaches. In this chapter we provide an (updated) taxonomy of risk-based testing aligned with risk considerations in all phases of a test process. It consists of three top-level classes: contextual setup, risk assessment, and risk-based test strategy. This taxonomy provides a framework to understand, categorize, assess, and compare risk-based testing approaches to support their selection and tailoring for specific purposes. Furthermore, we position four recent risk-based testing approaches into the taxonomy in order to demonstrate its application and alignment with available risk-based testing approaches.

In order to optimize the usage of testing efforts and to assess risks of software-based systems, risk-based testing uses risk (re-)assessments to steer all phases in a test process. Several risk-based testing approaches have been proposed in academia and/or applied in industry, so that the determination of principal concepts and methods in risk-based testing is needed to enable a comparison of the weaknesses and strengths of different risk-based testing approaches. In this chapter we provide an (updated) taxonomy of risk-based testing aligned with risk considerations in all phases of a test process. It consists of three top-level classes, i.e., contextual setup, risk assessment, and risk-based test strategy. This taxonomy provides a framework to understand, categorize, assess and compare risk-based testing approaches to support their selection and tailoring for specific purposes. Furthermore, we position four recent risk-based testing approaches into the taxonomy in order to demonstrate its application and alignment with available risk-based testing approaches.

Managing cyber security has become increasingly important due to the growing interconnectivity of computerized systems and their use in society. A comprehensive assessment of cyber security can be challenging as its spans across different domains of knowledge and expertise. For instance, identifying cyber security vulnerabilities requires detailed technical expertise and knowledge, while the assessment of organizational impact and legal implications of cyber security incidents may require expertise and knowledge related to risk and compliance. Standards like ISO 31000 and ISO/IEC/IEEE 29119 detail the relevant aspects of risk management and testing and thus provide guidance in these areas. However, both standards are not exclusively dedicated to the subject of security and do not cover the explicit integration between security risk assessment and security testing. We think however, that they provide a good basis for that. In this paper we show how ISO 31000 and ISO/IEC/IEEE 29119 can be integrated to provide a comprehensive approach to cyber security that covers both security risk assessment and security testing.

Managing cyber security has become increasingly important due to the growing interconnectivity of computerized systems and their use in society. A comprehensive assessment of cyber security can be challenging as its spans across different domains of knowledge and expertise. For instance, identifying cyber security vulnerabilities requires detailed technical expertise and knowledge, while the assessment of organizational impact and legal implications of cyber security incidents may require expertise and knowledge related to risk and legal compliance. We present a method that provides a comprehensive approach to cyber security by integrating three areas of cyber security assessment which are traditionally viewed in isolation: risk assessment, security testing, and legal compliance.

David Hutchison Lancaster University, Lancaster, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Zürich, Switzerland John C. Mitchell Stanford University, Stanford, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Dortmund, Germany Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany

Complex networked systems are an integral part of today's support infrastructures. Due to their importance, these systems become more and more the target for cyber-attacks, suffering a notable number of security incidents. Also, they are subject to regulation by national and international legislation. An operator of such an infrastructure or system is responsible for ensuring its security and correct functioning in order to satisfy customers. In addition, the entire process of risk and quality control needs to be efficient and manageable. This short paper introduces the Compliance, Risk Assessment and Security Testing Improvement Profiling (CRSTIP) scheme. CRSTIP is an evaluation scheme that enables assessing the maturity of security assessment processes, taking into consideration systematic use of formalisms, integration and tool-support in the areas of compliance assessment, security risk assessment and security testing. The paper describes the elements of the scheme and their application to one of the case studies of the RASEN research project.

A testing language typically provides a set of test automation statements that allows for a systematic definition and automatic application of stimulation data (i.e. messages or signals) to a system under test. Moreover, it eases the assessment of the system’s reaction by providing customizable evaluation statements and functions. TTCN-3 , the Testing and Test Control Notation, already provides universal and powerful concepts to describe tests for discrete, message-based systems. However, software-based control systems that are used to control physical processes often show continuous quantities that can be only poorly stimulated and assessed by means of the currently available language constructs in TTCN-3 . In this article, we show how this problem can be solved by extending the TTCN-3 language. We introduce an extension of TTCN-3 , namely TTCN-3 embedded , that provides concepts and constructs that directly address the specification of tests for continuous and hybrid real time systems. The extension includes the notion of streams that can be used to represent continuous quantities over time. In addition, TTCN-3 has been extended with the concepts of stream-based ports, sampling, equation systems, and with additional control flow structures. The concepts are integrated with standard TTCN-3 and allow for defining test cases that handle continuous quantities, as well as discrete state changes and the exchange of messages within the same concept space. The feasibility of the approach is shown by providing a small example from the automotive industry.

This book constitutes the thoroughly refereed conference proceedings of the First International Workshop on Risk Assessment and Risk-driven Testing, RISK 2013, held in conjunction with 25th IFIP Inter

Recently, in the Security testing domain a lot of knowledge has been collected from a significant amount of research. The contribution provides an introduction to advanced security testing methods and techniques in the context of European research and standardization projects. This includes numerous guidelines and best practices that have been identified and are applied in the context of industrial case studies. In particular it addresses risk modeling, security test pattern, functional security tests as well as fuzz testing, as important contributions to systematic, automatized test approaches in research, industry and standardization.