In this paper, we propose PATO-a producibility-aware topology optimization (TO) framework to help efficiently explore the design space of components fabricated using metal additive manufacturing (AM), while ensuring manufacturability with respect to cracking. Specifically, parts fabricated through Laser Powder Bed Fusion are prone to defects such as warpage or cracking due to high residual stress values generated from the steep thermal gradients produced during the build process. Maturing the design for such parts and planning their fabrication can span months to years, often involving multiple handoffs between design and manufacturing engineers. PATO is based on the a priori discovery of crack-free designs, so that the optimized part can be built defect-free at the outset. To ensure that the design is crack free during optimization, producibility is explicitly encoded within the standard formulation of TO, using a crack index. Multiple crack indices are explored and using experimental validation, maximum shear strain index (MSSI) is shown to be an accurate crack index. Simulating the build process is a coupled, multi-physics computation and incorporating it in the TO loop can be computationally prohibitive. We leverage the current advances in deep convolutional neural networks and present a high-fidelity surrogate model based on an Attention-based U-Net architecture to predict the MSSI values as a spatially varying field over the part's domain. Further, we employ automatic differentiation to directly compute the gradient of maximum MSSI with respect to the input design variables and augment it with the performance-based sensitivity field to optimize the design while considering the trade-off between weight, manufacturability, and functionality. We demonstrate the effectiveness of the proposed method through benchmark studies in 3D as well as experimental validation.

Artificial Intelligence/Machine Learning (AI/ML) is a growing field that has potential for widespread usage in the aerospace industry. However, the traditional process-based approaches for aerospace systems certification fall short of addressing the uncertainties and complexities associated with AI/ML technologies. This paper presents the results of applying a novel Overarching Properties (OP)-based approach for the assurance of complex digital aerospace systems that contain AI/ML-based subcomponents. The OPs are being evaluated by the FAA and NASA as a foundation for developing an alternate means of compliance (MOC) for the certification of aerospace systems. The OP-based approach evaluated in this paper uses structured premise-based arguments, where the premises are designed to address the different aspects of AI/ML technologies with respect to system-level safety and design needs. The structured arguments align the low-level properties of AI/ML components to system-level properties by using the three OPs labeled - Intent, Correctness, and Innocuity - making it easy to logically establish the safety and correctness of AI/ML-based digital aerospace systems. We use a Recorder Independent Power Supply (RIPS) example, that contains AI/ML-based components, to evaluate the OP-based assurance approach. We describe in detail the process of generating verification and validation evidence to support the arguments and presenting the evidence using assurance cases.

Many works have been focused in developing detection, monitoring and prediction routines for asset health estimation system. Classic machine learning based models benefit from quality of physics-informed features available from domain knowledge. This, however, can be labor intensive and is limited by quality of features developed through available knowledge. Deep learning based approach, if successful, can alleviate this laborious step. On the other hand, users often need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. In this work, we propose a deep learning based regression network that output both prediction value and confidence score for asset health estimation in short intermittent transients time series application. In the experimental study, we show that our model has low prediction error given short intermittent transients multivariate time series as input. Furthermore, our model also provides a confidence score for each prediction that is highly negatively correlated with true prediction error. Experiments show that by setting an acceptance threshold on confidence score, our model can reach an averaged improvement of 20% on the prediction quality with 90% coverage.

Performance of a Prognostics and Health Management (PHM) system in a fielded application depends on observability from existing monitoring equipment and sensing, which get determined at the design phase. Although various technologies have been proposed in the literature, there is currently a lack of known generic tools specifically designed for performing design stage sensor placement analyses from a PHM perspective. This leads to PHM observability being an afterthought and resulting PHM designs being sub-optimal. This paper describes a new Optimal Sensor Placement (OSP) framework, its implementation as a toolkit and the experience with applying it to a new product design in the context of a Small Modular Reactor (SMR). The formulation adds multiple important features that are critical to PHM applications. Firstly, it establishes a direct link to PHM performance requirements with intent to reduce operational and maintenance costs. Moreover, it acknowledges and accounts for the costs and risks of errors that PHM system will incur, and simultaneously considers operational requirements on sensing for performance, control and/or regulatory requirements. The toolkit described here implements formulations of a large number of requirements scenarios applicable in a generic industrial product development setting.

The traditional process-based approaches to certifying aerospace digital systems are not sufficient to address the challenges associated with using Artificial Intelligence (AI) or Machine Learning (ML) techniques. To address this, agencies like the National Aeronautics and Space Administration (NASA) and the Federal Aviation Administration (FAA) are evaluating an alternative Means of Compliance (MoC) called the Overarching Properties (OP). We propose a novel foundation for the application of OPs to support the certification of complex aerospace digital systems consisting of AI/ML-based components. Our approach utilizes well-defined argument structures, that are justified by premises specialized to the AI/ML domain, to logically claim that an AI/ML-based component will possess the OPs. To motivate our work, we execute the design process of a Recorder Independent Power Supply (RIPS) system that provides several minutes of backup power to the data recorder when an aircraft loses access to its standard power supply.

Free-form text-based maintenance and service records related to industrial assets capture the observations and actions of service engineers and are a crucial resource for assessing system-level asset health. To facilitate tracking of historical asset health issues, these records are categorized using tags from a predefined taxonomy, which is mostly a manual and time-consuming process. Given that these records can offer valuable information in troubleshooting maintenance issues, automating this process through deep learning (DL) based natural language processing (NLP) models can offer significant operational and maintenance (O&M) cost reductions. However, these data-based models are not expected to be fully accurate, requiring human experts to regularly review all predictions by DL models to verify or correct them, which is also an highly inefficient and costly process. On the other hand, new records that have novel or ambiguous context can be more appropriately resolved by a human expert. The objective of the work described in this paper is to create an interpretable mechanism that can assess reliability of individual predictions from DL-based maintenance record classifiers and help design a mixed initiative system. This system aims to identify scenarios where predictions are reliable enough for automated decision versus where human intervention is needed due to poor reliability. Additionally, this system aides decision support by providing exemplars from training set that can enhance the human tagger’s productivity and quality. Given a set of tagged records, it also has the capability to identify instances where the originally assigned tags are likely to be inaccurate/noisy. We illustrate these outcomes through tagging of maintenance records from the aviation domain, leading to improvements over only human-based or only DL-based tag assignments.

Recent work has demonstrated robust mechanisms by which attacks can be orchestrated on machine learning models. In contrast to adversarial examples, backdoor or trojan attacks embed surgically modified samples in the model training process to cause the targeted model to learn to misclassify samples in the presence of specific triggers, while keeping the model performance stable across other nominal samples. However, current published research on trojan attacks mainly focuses on classification problems, which ignores sequential dependency between inputs. In this paper, we propose methods to discreetly introduce and exploit novel backdoor attacks within a sequential decision-making agent, such as a reinforcement learning agent, by training multiple benign and malicious policies within a single long short-term memory (LSTM) network, where the malicious policy can be activated by a short realizable trigger introduced to the agent. We demonstrate the effectiveness through initial outcomes generated from our approach as well as discuss the impact of such attacks in defense scenarios. We also provide evidence as well as intuition on how the trojan trigger and malicious policy is activated. In the end, we propose potential approaches to defend against or serve as early detection for such attacks.

The rapid and accurate prediction of residual stresses in metal additive manufacturing (3D printing) processes is crucial to ensuring defect-free fabrication of parts used in critical industrial applications. This paper presents promising outcomes from applying attention-based neural architectures for predicting such 3D stress phenomena accurately, efficiently, and reliably. This capability is critical to drastically reducing the design maturation time for additively manufactured parts. High fidelity, physics-based numerical models of the additive melting process exist that can simulate the thermal gradients and consequent stresses produced during manufacturing, which can then be used to synthesize a 3D crack index field for the entire part volume, capturing the likelihood that a region in a part will crack upon heat treatment. However, these models are expensive and time-consuming to run. In response, a Deep Convolutional Neural Network (DCNN) model is explored as a surrogate for the physics-based model, so that it can be used to time-efficiently estimate the crack index for a given part-design. This requires careful design of the training regime and dataset for a given design problem. Using the U-Net architecture as the baseline, we expand the standard 2D application of this architecture for segmentation to the estimation of the full 3D, continuous valued, stress field. We illustrate the primary challenge faced by the standard U-Net architecture with L2-loss arising from sparsity in critical values of the crack index and show how augmenting the architecture with attention mechanisms helps address the issue as well as improve the overall accuracy of estimation.

We propose a novel framework for real-time black-box universal attacks which disrupts activations of early convolutional layers in deep learning models. Our hypothesis is that perturbations produced in the wavelet space disrupt early convolutional layers more effectively than perturbations performed in the time domain. The main challenge in adversarial attacks is to preserve low frequency image content while minimally changing the most meaningful high frequency content. To address this, we formulate an optimization problem using time-scale (wavelet) representations as a dual space in three steps. First, we project original images into orthonormal sub-spaces for low and high scales via wavelet coefficients. Second, we perturb wavelet coefficients for high scale projection using a generator network. Third, we generate new adversarial images by projecting back the original coefficients from the low scale and the perturbed coefficients from the high scale sub-space. We provide a theoretical framework that guarantees a dual mapping from time and time-scale domain representations. We compare our results with state-of-the-art black-box attacks from generative-based and gradient-based models. We also verify efficacy against multiple defense methods such as JPEG compression, Guided Denoiser and Comdefend. Our results show that wavelet-based perturbations consistently outperform time-based attacks thus providing new insights into vulnerabilities of deep learning models and could potentially lead to robust architectures or new defense and attack mechanisms by leveraging time-scale representations.

Machine learning models provide statistically impressive results which might be individually unreliable. To provide reliability, we propose an Epistemic Classifier (EC) that can provide justification of its belief using support from the training dataset as well as quality of reconstruction. Our approach is based on modified variational auto-encoders that can identify a semantically meaningful low-dimensional space where perceptually similar instances are close in $\ell_{2}-$distance too. Our results demonstrate improved reliability of predictions and robust identification of samples with adversarial attacks as compared to baseline of softmax-based thresholding.

At GE Research, we are combining “physics” with artificial intelligence and machine learning to advance manufacturing design, processing, and inspection, turning innovative technologies into real products and solutions across our industrial portfolio. This article provides a snapshot of how this physical plus digital transformation is evolving at GE.

Inflow Control Devices (ICDs) help reduce the adverse consequences of uneven inflow issues in a lateral completion system. The most common uneven inflow consequences are early water breakthrough and gas coning in water-driven and saturated reservoirs. These issues lead to the dominance of undesired fluid production and consequently, reduced well productivity. Typically, uneven inflow issues are caused by different drivers, including heterogenous permeability, an uneven water saturation profile, and/or complex well completion in a lateral section of a given well. ICDs are placed in permanent positions along the lateral section of a well in order to control zonal production and improve well productivity. The goal of utilizing ICDs is to delay water or gas production and equalize the inflow production from the reservoir to wellbore. However, the uncertainty of reservoir characteristics and operational constraints add complexity to the ICD design and complicate optimization strategies. An optimum ICD design entails identifying the number and size of compartments, packer locations, ICD type, and number of ICDs in each compartment, and the ICD settings such as orifice diameter or flow restriction rating. Extensive reservoir modeling work can be performed to accurately quantify the impact of each ICD design on well production. The intent of this paper is to demonstrate that Bayesian optimization and machine learning techniques can help identify an optimized ICD design in a minimum number of reservoir simulation evaluations. These techniques are implemented into the reservoir simulation workflow to enhance the speed of the analysis and resulting value proposition for the operating customer.Using Gaussian Process Regression as a surrogate, Bayesian optimization makes use of a small number of initial reservoir simulation runs to quantify the uncertainty of the surrogate model in the parameter space. It makes use of an appropriate acquisition function (as determined by the desired exploration-exploitation tradeoff characteristics) to design the next sample (simulation run) to be evaluated. Unlike the ensemble-based optimization algorithms, Bayesian optimization points to the optimum solution sequentially (one evaluation at a time). The proposed workflow automates the optimization process of ICD design evaluation workflow times by 50% in our case studies. The 50% efficiency takes in the time to perform ICD optimization workflow. For instance, the manual iteration ICD design for case study 1 described in this paper was four weeks, which the proposed workflow shortened this time to two weeks.This paper presents two case studies in which the Bayesian optimization technique was used to identify the best ICD completion design. The space parameter in both case studies involves several variables, including the number and location of compartments, the number of ICDs per compartment, and the ICD settings (one such setting, for example, considers orifice diameter size). The goal in the first case study was to find an ICD design that can maximize the net present value over the well lifetime (set to 5 years), while reducing and delaying water production. In this first case study, an 800ft lateral in a horizontal well, with drastic variation of permeability along its lateral length, was considered. In the second case study, 4000ft horizontal length of a well with variations of permeability was analyzed. In this second case, the objective was to extend the life of the well by minimizing the gas-oil ratio and maximizing the oil recovery. The simulation runs stopped after 3 years of production and the best case was chosen based on the aforementioned criteria. In both case studies, the optimization algorithm setup was able to converge to an optimum ICD design within 20 reservoir simulation runs. This alone represents an improvement over the current manual trial and error process in which an expert uses human intuition.

With the advent of Deep Learning, the field of machine learning (ML) has surpassed human-level performance on diverse classification tasks. At the same time, there is a stark need to characterize and quantify reliability of a model's prediction on individual samples. This is especially true in applications of such models in safety-critical domains of industrial control and healthcare. To address this need, we link the question of reliability of a model's individual prediction to the epistemic uncertainty of the model's prediction. More specifically, we extend the theory of Justified True Belief (JTB) in epistemology, created to study the validity and limits of human-acquired knowledge, towards characterizing the validity and limits of knowledge in supervised classifiers. We present an analysis of neural network classifiers linking the reliability of its prediction on a test input to characteristics of the support gathered from the input and hidden layers of the network. We hypothesize that the JTB analysis exposes the epistemic uncertainty (or ignorance) of a model with respect to its inference, thereby allowing for the inference to be only as strong as the justification permits. We explore various forms of support (for e.g., k-nearest neighbors (k-NN) and ℓp-norm based) generated for an input, using the training data to construct a justification for the prediction with that input. Through experiments conducted on simulated and real datasets, we demonstrate that our approach can provide reliability for individual predictions and characterize regions where such reliability cannot be ascertained.

In machine learning, backdoor or trojan attacks during model training can cause the targeted model to deceptively learn to misclassify in the presence of specific triggers. This mechanism of deception enables the attacker to exercise full control on when the model behavior becomes malicious through use of a trigger. In this paper, we introduce Epistemic Classifiers as a new category of defense mechanism and show their effectiveness in detecting backdoor attacks, which can be used to trigger default mechanisms, or solicit human intervention, on occasions where an untrustworthy model prediction can adversely impact the system within which it operates. We show experimental results with multiple public datasets and explain the reasons with visualization for effectiveness of the proposed approach. This empowers the war fighter to trust the AI on the tactical edge to be reliable and to become sensitive to scenarios with deception and noise where reliability cannot be provided.

Abstract Optimization problems, such as optimal well-spacing or completion design, can be resolved rapidly via surrogate proxy models, and these models can be built using either data-based or physics-based methods. Each approach has its strengths and weaknesses with respect to management of uncertainty, data quality or validation. This paper explores how data- and physics-based proxy models can be used together to create a workflow that combines the strengths of each approach and delivers an improved representation of the overall system. This paper presents use cases that display reduced simulation computational costs and/or reduced uncertainty in the outcomes of the models. A Bayesian calibration technique is used to improve predictability by combining numerical simulations with data regressions. Discrepancies between observations and surrogate outcomes are then observed to calibrate the model and improve the prediction quality and further reduce uncertainty. Furthermore, Gaussian Process Regression is used to locate global minima/maxima, with a minimal number of samples. To demonstrate the methodology, a reservoir model involving two wells in a drill space unit (DSU) in the Bakken Formation was constructed using publicly available data. This reservoir model was tuned by history matching the production data for the two wells. A data-based regression model was constructed based on machine learning technologies using the same dataset. Both models were coupled in a system to build a hybrid model to test the proposed process of data and physics coupling for completion optimization and uncertainty reduction. Subsequently, Gaussian Process Model was used to explore optimization scenarios outside of the data region of confidence and to exploit the hybrid model to further reduce uncertainty and prediction. Overall, both the computation time to identify optimal completion scenarios and uncertainty were reduced. This technique creates a robust framework to improve operational efficiency and drive completion optimization in an optimal timeframe. The hybrid modeling workflow has also been piloted in other applications such as completion design, well placement and optimization, parent-child well interference analysis, and well performance analysis.

Recent work has demonstrated robust mechanisms by which attacks can be orchestrated on machine learning models. In contrast to adversarial examples, backdoor or trojan attacks embed surgically modified samples with targeted labels in the model training process to cause the targeted model to learn to misclassify chosen samples in the presence of specific triggers, while keeping the model performance stable across other nominal samples. However, current published research on trojan attacks mainly focuses on classification problems, which ignores sequential dependency between inputs. In this paper, we propose methods to discreetly introduce and exploit novel backdoor attacks within a sequential decision-making agent, such as a reinforcement learning agent, by training multiple benign and malicious policies within a single long short-term memory (LSTM) network. We demonstrate the effectiveness as well as the damaging impact of such attacks through initial outcomes generated from our approach, employed on grid-world environments. We also provide evidence as well as intuition on how the trojan trigger and malicious policy is activated. Challenges with network size and unintentional triggers are identified and analogies with adversarial examples are also discussed. In the end, we propose potential approaches to defend against or serve as early detection for such attacks. Results of our work can also be extended to many applications of LSTM and recurrent networks.

Abstract Frac hits are a form of fracture-driven interference (FDI) that occur when newly drilled wells communicate with existing wells during completion, and which may negatively or positively affect production. An analytics and machine-learning approach is presented to characterize and aid understanding of the root causes of frac hits. The approach was applied to a field data set and indicated that frac hits can be quantitatively attributed to operational or subsurface parameters such as spacing or depletion. The novel approach analyzed a 10-well pad comprising two ‘parent’ producers and eight ‘child’ infills. The analysis included the following data types: microseismic, completion, surface and bottomhole pressure, tracers, production, and petrophysical logs. The method followed a three-step process: 1) use analytics to assess interference during the hydraulic fracturing and during production, 2) catalogue or extract feature engineering attributes for each stage (offset distance, petrophysics, completion, and depletion) and 3) apply machine-learning techniques to identify which attributes (operations or subsurface) are significant in the causation and/or enhancement of inter-well communication. Information fusion with multi-modal data was also used to determine the probability of well-to-well communication. The data fusion technique integrated multiple sensor data to obtain a lower detection error probability and a higher reliability by using data from multiple sources. The results showed that the infill wells completed in closest proximity to the depleted parents exhibit strong communication. The machine-learning classification creates rules that enable better understanding of control variables to improve operational efficiency. Furthermore, the methodology lends a framework that enables the development of visualization, continuous learning, and real-time application to mitigate communication during completions.

Abstract Given limited CO2 supply, operational constraints, and pattern specific reservoir performance, WAG schedule can be customized such that NPV or other metrics are optimized. Depending on the WAG schedule, recovery can fluctuate between 5–15% at the pattern scale due to reservoir heterogeneity causing variations in sweep efficiency. An analytical method was developed to optimize WAG schedules that couples traditional reservoir modeling and simulation with machine learning, enabling the discovery of optimal WAG schedules that increase recovery at the pattern level. A history-matched reservoir model of Chaparral Energy's Farnsworth Field, Ochiltree County, TX was sampled intelligently to perform predictive reservoir flow simulations and artificially build an intelligent reservoir model that samples a broad range of possible WAG scenarios for optimization. The intelligent model generates the next "best" sample to investigate in the numerical simulator and converges on the optima, quickly reducing the number of runs investigated. Results in this paper demonstrate that there can be significant improvements in net present value as well as net utilization rates of CO2 using this analytical technique. The WAG design generated by the intelligent reservoir model should be deployed in the field in early 2016 for validation. It is intended that the intelligent reservoir model will be updated on a regular basis as injection and production data is obtained. This effort represents the beginning of a paradigm shift in the application of modeling and simulation tools for significant improvements in field production operations.