The DoD desires to incorporate autonomous systems into its capabilities. These systems are notoriously difficult to validate, though they require assurance to be used. One area of interest is Intelligence, Surveillance and Reconnaissance (ISR), whose tasks are typically written in natural language. Temporal logics, e.g., Linear Temporal Logic (LTL) and Signal Temporal Logic (STL), have been used in robotics, cyber-physical systems, and electronics design domains to provide confidence and assurance in the correctness, reliability, and safety of system designs. ISR tasks written as temporal logic specifications enables their properties to be validated. We found that recent natural language to LTL/STL specification translators incorrectly translate natural language ISR descriptions. In this paper, we provide manual translations of ten ISR tasks to temporal logic and validate the specifications. We believe this is the first research to provide a set of ISR tasks specified in temporal logic and checked by a satisfiability solver.

We study the performance of BLOOM large language models of different sizes on understanding 12 different languages from the Chomsky hierarchy using few-shot prompts. We investigate whether an increase in the complexity of the languages learned by the larger models can be characterized using the Chomsky hierarchy. We first show that prompting in BLOOM models enables reasoning with a good accuracy on language tasks as diverse as stack manipulation, string reversal, odds first, and interlocked pairing, when the queries are over short strings, that is, small bitwidth bit-vectors from the language. Second, we discover that the two largest models have the highest accuracy on such tasks for prompts with a fixed length, but smaller models are able to achieve similar accuracies with longer prompts. Unlike classical automata or grammar based approaches where algorithms for more complex languages in the Chomsky hierarchy can also recognize simpler languages, we find that the performance of the BLOOM large language models cannot be explained by the complexity of the languages in the Chomsky hierarchy.

Health information technology (HIT) was introduced to streamline administrative medical processes and alleviate errors that may negatively affect patient health. It has helped to some extent in this manner; however, studies have shown that the use of HIT has introduced unexpected new errors in the collection, transmission, use, and processing of electronic health records, as well as potential programming or configuration errors in these HIT systems. This study was initiated to identify HIT anomalies and, from this, potential hazards that may threaten patient safety. The results can be used to identify and prioritize hazard classes, and to assist in the design and implementation of HIT hazard controls and detectors.

Reinforcement learning (RL) has received significant interest in recent years, primarily because of the success of deep RL in solving many challenging tasks, such as playing chess, Go, and online computer games. However, with the increasing focus on RL, applications outside gaming and simulated environments require an understanding of the robustness, stability, and resilience of RL methods. To this end, we conducted a comprehensive literature review to characterize the available literature on these three behaviors as they pertain to RL. We classified the quantitative and theoretical approaches used to indicate or measure robustness, stability, and resilience behaviors. In addition, we identified the actions or events to which the quantitative approaches attempted to be stable, robust, or resilient. Finally, we provide a decision tree that is useful for selecting metrics to quantify behavior. We believe that this is the first comprehensive review of stability, robustness, and resilience, specifically geared toward RL.

Responsible integration of deep neural networks into the design of trustworthy systems requires the ability to explain decisions made by these models. Explainability and transparency are critical for system analysis, certification, and human-machine teaming. We have recently demonstrated that neural stochastic differential equations (SDEs) present an explanation-friendly DNN architecture. In this paper, we present ExplainIt, an online tool for explaining AI decisions that uses neural SDEs to create visually sharper and more robust attributions than traditional residual neural networks. Our tool shows that the injection of noise in every layer of a residual network often leads to less noisy and less fragile integrated gradient attributions. The discrete neural stochastic differential equation model is trained on the ImageNet data set with a million images, and the demonstration produces robust attributions on images in the ImageNet validation library and on a variety of images in the wild. Our online tool is hosted publicly for educational purposes.

In highly configurable health information technology (HIT) systems, such as VistA of the Veterans Health Administration, the variations in how the system is used among different healthcare facilities and how the data are recorded can be significant. Despite the successful standardization of care efforts, some of these variations can be indicative of HIT hazards and demand further investigation. In this work, we implemented a recurrent neural network (RNN) architecture to learn clinical provider order sequences and their temporal dynamics while predicting the orders' terminal state. We demonstrate model performance and provide a use case for the model discerning novel event sequences. This model is proposed to find novel event sequences in an operational environment.

The adoption of health information technology (HIT) has facilitated efforts to increase the quality and efficiency of health care services and decrease health care overhead while simultaneously generating massive amounts of digital information stored in electronic health records (EHRs). However, due to patient safety issues resulting from the use of HIT systems, there is an emerging need to develop and implement hazard detection tools to identify and mitigate risks to patients. This paper presents a new methodological framework to develop hazard detection models and to demonstrate its capability by using the US Department of Veterans Affairs' (VA) Corporate Data Warehouse, the data repository for the VA's EHR. The overall purpose of the framework is to provide structure for research and communication about research results. One objective is to decrease the communication barriers between interdisciplinary research stakeholders and to provide structure for detecting hazards and risks to patient safety introduced by HIT systems through errors in the collection, transmission, use, and processing of data in the EHR, as well as potential programming or configuration errors in these HIT systems. A nine-stage framework was created, which comprises programs about feature extraction, detector development, and detector optimization, as well as a support environment for evaluating detector models. The framework forms the foundation for developing hazard detection tools and the foundation for adapting methods to particular HIT systems.

Autonomous systems are complex, diverse, and likely be used in environments that change unpredictably. We must develop mechanisms to cope with failures, potentially caused by both the natural stresses of the operating environment and the system's inadequate environmental models. Failures also arise from system component wear. In 2016, the IEEE launched the Global Initiative on Ethics of Autonomous and Intelligent Systems in response to the proliferation of increasingly-autonomous systems. A cornerstone of this initiative is the IEEE P7000 series of projects, which set the standards for the future of ethical intelligent and autonomous technologies. This paper describes the ongoing development of the IEEE P7009 standard on “Fail-Safe Design of Autonomous and Semi-Autonomous Systems”. We provide a collective perspective on the ongoing development of IEEE P7009, and outline the objectives, current approaches, issues under discussion, and two exemplar use cases from different sectors: one high-regulation, the other low-regulation.

Nondestructive evaluation methods play an important role in ensuring component integrity and safety in many industries. Operator fatigue can play a critical role in the reliability of such methods. This is important for inspecting high value assets or assets with a high consequence of failure, such as aerospace and nuclear components. Recent advances in convolution neural networks can support and automate these inspection efforts. This paper proposes using residual neural networks (ResNets) for real-time detection of corrosion, including iron oxide discoloration, pitting and stress corrosion cracking, in dry storage stainless steel canisters housing used nuclear fuel. The proposed approach crops nuclear canister images into smaller tiles, trains a ResNet on these tiles, and classifies images as corroded or intact using the per-image count of tiles predicted as corroded by the ResNet. The results demonstrate that such a deep learning approach allows to detect the locus of corrosion via smaller tiles, and at the same time to infer with high accuracy whether an image comes from a corroded canister. Thereby, the proposed approach holds promise to automate and speed up nuclear fuel canister inspections, to minimize inspection costs, and to partially replace human-conducted onsite inspections, thus reducing radiation doses to personnel.

Background and Purpose Process mining for conformance analysis consists of comparing a reference process model against a data-driven process model generated via log files from information technology systems. However, in the absence of a complete reference process model, we found no suggested approaches in the literature to address the need for evaluating process conformance among different healthcare facilities to assess standardization of care. Our goal is to find similarities and dissimilarities in data-driven process models among US Veterans Health Administration (VHA) facilities that can be indicative of patient safety issues. Our hypothesis was that the analysis would not produce statistically significant differences in outcome.Methods We present a unique implementation of conformance analysis in process mining that consists of combining process mining, process mapping and statistical metrics. We illustrate our approach by applying it to the analysis of two clinical radiology order process models generated from healthcare data provided by two similar facilities in the VHA.Results The comparative assessment showed that about 70% of the orders completed successfully and 30% were not completed due to policy and duplications. Our analysis found a good statistical correlation between both facilities, as the Spearman’s correlation coefficient between facilities for the frequency of cases per total hours was 0.87879, for the frequency of cases by state transition was 0.79702 and for the throughput time per state transition was 0.63582. Additional statistical analyses using the Mann-Whitney U test and the root mean square error both produced values that were not significant.Conclusions The foregoing approach validated our hypothesis by demonstrating a good statistical correlation of data describing the flow of clinical radiology orders absent a credible reference model. Finding good agreement between both facilities was important in confirming that the clinical orders flow in a similar manner, suggesting standardization of care.

Over the past decade, health information technology (IT) has enabled the amount of digital information stored in electronic health records (EHRs) to expand greatly. However, according to some studies, hazards in health IT can lead to changes in clinical decisions, care processes, and care outcomes, as well as other issues. Thus, the effects of health IT hazards on patient safety have been at the forefront of recent patient safety research. Nonetheless, hazard detection in health IT remains a challenge. In this paper, the authors assume that safety-related issues in health IT would exhibit anomalous characteristics in EHR data. Although all hazards will exhibit some anomalous characteristics, not all anomalies can be regarded as hazards. The authors hypothesize that errors in health IT could lead to interruptions in the sequence of clinical actions. To this end, the problem of detecting anomalous sequences in big EHR data is considered. This paper focuses on dynamic event sequences, which are a series of clinical actions in motion. The authors propose an adaptive anomaly detection approach that uses higher-order network representation to detect anomalous sequences. Furthermore, the authors propose a contiguous subsequence anomaly detection approach that identifies abnormal subsequences in the detected anomalous sequences. The proposed approaches are tested by using synthetic and real-world EHR data. The proposed methods outperform existing state of the art anomaly detection techniques. To reduce the computational complexity associated with the operational implementation of the proposed approaches, the Apache Spark environment was leveraged, and a much shorter run time together with improved performance were achieved, especially for data with more than 60,000 sequences.

Deep neural networks have been shown to be vulnerable to membership inference attacks wherein the attacker aims to detect whether specific input data were used to train the model. These attacks can potentially leak private or proprietary data. We present a new extension of Fano's inequality and employ it to theoretically establish that the probability of success for a membership inference attack on a deep neural network can be bounded using the mutual information between its inputs and its activations. This enables the use of mutual information to measure the susceptibility of a DNN model to membership inference attacks. In our empirical evaluation, we show that the correlation between the mutual information and the susceptibility of the DNN model to membership inference attacks is 0.966, 0.996, and 0.955 for CIFAR-10, SVHN and GTSRB models, respectively.

Given a contact network and coarse-grained diagnostic information such as electronic Healthcare Reimbursement Claims (eHRC) data, can we develop efficient intervention policies from data to control an epidemic? Immunization is an important problem in multiple areas, especially epidemiology and public health. However, most existing studies rely on assuming prior epidemiological models to develop pre-emptive strategies, which may fail to adapt to the change in new epidemiological patterns and the availability of rich data such as eHRC. In practice, disease spread is usually complicated, hence assuming an underlying model may deviate from true spreading patterns, leading to possibly inaccurate interventions. Additionally, the abundance of health care surveillance data (such as eHRC) makes it possible to study data-driven strategies without too many restrictive assumptions. Hence, such a data-driven intervention approach can help public-health experts take more practical decisions. In this paper, we take into account propagation log and contact networks for controlling propagation. Different from previous model-based approaches, our solutions are solely data driven in a sense that we develop immunization strategies directly from the network and eHRC without assuming classical epidemiological models. In particular, we formulate the novel and challenging data-driven immunization problem. To solve it, we first propose an efficient sampling approach to align surveillance data with contact networks, then develop an efficient algorithm with the provably approximate guarantee for immunization. Finally, we show the effectiveness and scalability of our methods via extensive experiments on multiple datasets, and conduct case studies on nation-wide real medical surveillance data.

Deep learning systems have achieved great success in various types of applications over recent years. They are increasingly being adopted for safety critical tasks, such as face recognition, surveillance systems, speech recognition, and autonomous driving. On the other hand, it has been found that deep neural networks (DNNs) can be easily fooled by adversarial input samples. These imperceptible perturbations on images can lead any machine learning system to misclassify the objects with high confidence. Furthermore, they can be almost indistinguishable to a human observer. These systems can also be exposed to adverse weather conditions such as fog, rain, and snow. This vulnerability raises major concerns in security-sensitive environments. Therefore, vulnerability of deep learning systems to synthetic adversarial attacks has been extensively studied and demonstrated, but the impact of natural weather conditions on these systems has not been studied in detail.

This paper presents an application of topic modeling on event sequences of Electronic Health Record (EHR) orders. Metaphorically, we approach clinical order event sequences of unlabeled data as if they are documents where words are the events that occurred in the history of an order. We demonstrate the approach leveraging Consult orders data. The details of the data preprocessing and the data structures are described along with the data sources. Latent Dirichlet Allocation (LDA) is leveraged to fit against the limited datasets prepared. Another open source tool - LDAvis is used for exploratory analysis of the LDA results. The preliminary results revealed some order patterns that are qualitatively evaluated as potential irregular transitions. The goal of this analysis is to provide unsupervised learning application to domain experts in the absence of labeled data where they can investigate captured patterns and identify irregular transitions of orders. Ultimately, such efforts will guide formalization of hazard detection algorithms that monitor EHR data to identify health information technology related hazards.

MET is a relatively new workshop on metamorphic testing for academic researchers and industry practitioners. The first international workshop on MET (MET 2016) was co-located with the 38th International Conference on Software Engineering (ICSE 2016) in Austin TX, USA on May 16, 2016. Since then the workshop has become an annual event. This paper reports on the fourth International Workshop on Metamorphic Testing (MET 2019) held in Montréal, Canada on May 26, 2019, as part of the 41st International Conference on Software Engineering (ICSE 2019). We first outline the aims of the workshop, followed by a discussion of its keynote speech and technical program.

AbstractMET is a relatively new workshop on metamorphic testing for academic researchers and industry practitioners. The first international workshop on MET (MET 2016) was co-located with the 38th International Conference on Software Engineering (ICSE 2016) in Austin TX, USA on May 16, 2016. Since then the workshop has become an annual event. This paper reports on the fourth International Workshop on Metamorphic Testing (MET 2019) held in Montréal, Canada on May 26, 2019, as part of the 41st International Conference on Software Engineering (ICSE 2019). We first outline the aims of the workshop, followed by a discussion of its keynote speech and technical program.