Artificial Neural Networks (ANNs) are employed in many areas of industry such as pattern recognition, robotics, controls, medicine, and defence. Their learning and generalization capabilities make them highly desirable solutions for complex problems. However, they are commonly perceived as black boxes since their behavior is typically scattered around its elements with little meaning to an observer. The primary concern in safety critical systems development and assurance is the identification and management of hazards. The application of neural networks in systems where their failure can result in loss of life or property must be backed up with techniques to minimize these undesirable effects. Furthermore, to meet the requirements of many statutory bodies such as FAA, such a system must be certified. There is a growing concern in validation of such learning paradigms as continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure a reliable system performance. In this paper, we survey the application of neural networks in high assurance systems that have emerged in various fields, which include flight control, chemical engineering, power plants, automotive control, medical systems, and other systems that require autonomy. More importantly, we provide an overview of assurance issues and challenges with the neural network model based control scheme. Methods and approaches that have been proposed to validate the performance of the neural networks are outlined and discussed after a comparative examination.

Based on the artificial immune system,a novel artificial immune method was introduced into the tracking control of linear motor.Because of perfect learning,adaptability and efficient pattern recognition,the data process ability of linear motor was improved,which result in the improved position-precision and robustness of linear motor.The simulation verified the validity of the method.

Adding autonomic capabilities to network management systems provides great promise in delivering high QoS while lowering operation and maintenance cost. In this paper, we present a model-based approach to adding autonomic capabilities to a fault management system for cellular networks. We propose the use of modeling techniques to specify software failures and their dispositions at the model level for the target system. This facilitates the deployment of a control loop for adding autonomic capabilities into the system architecture, which include self-monitoring, self-healing, and self-adjusting. Our case study on the intelligent network fault management system illustrates the proposed approach by adding and deploying these autonomic capabilities derived from self-model specifications, to mitigate the risk of specified failures and maintain the level of healthiness of the system, dynamically and effectively.

Adding self-healing capabilities to network management systems holds great promise for delivering important goals, such as QoS, while simultaneously lowering capital expenditure, operation cost, and maintenance cost. In this paper, we present a model-based approach to add self-healing capabilities to a fault management system for cellular networks. We propose a generic modeling framework to categorize software failures and specify their dispositions at the model level for the target system. This facilitates the deployment of a control loop for adding autonomic capabilities into the system architecture, which include self-monitoring, self-healing, and self-adjusting functionality. While self- monitoring oversees the environmental conditions and system behavior, self-healing is accomplished by instrumenting the system with self-adjusting operations. We include a case study on a prototype intelligent network fault management system to illustrate this approach by showing how these autonomic capabilities can be added and deployed. Specifically, these autonomic capabilities are derived from self-model specifications, and are used to mitigate the risk of specified failures and maintain the health of the system in response to different types of faults encountered.

Self-healing is a vital property that an autonomic system must possess in order to provide robust performance and survivability. The promise of self-healing depends on other properties that the system should provide, which include self-monitoring and self-configuring. Autonomic systems further require self-healing behavior to adapt to changes in user needs, business goals, and environmental conditions such that self-healing decisions are made dynamically and adaptively according to the system context. In this paper, we propose a model-driven approach that leverages modeling techniques, reliability engineering methodologies, and aspect-oriented development to realize an adaptive self- healing paradigm for autonomic computing.

Traditional fixed-gain control has proven to be unsuccessful to deal with complex changing systems such as a damaged aircraft. Control systems, which use a neural network that can adapt toward changes in the plant, have been actively investigated and test flown as they offer many advantages. We will briefly introduce adaptive flight control and will discuss the specific challenges for the verification and validation (V&V) of such systems. Since performance and safety guarantees cannot be provided at development time, we have developed novel tools and approaches to support, V&V and certification, which use a Bayesian approach to monitor sensitivity and performance (confidence) of the neural network during flight.

Activity modeling is known as a powerful technique for designing and specifying the flow logic of a process. Due to the complexity of the described process, activity models may involve multiple activities that are tangled with each other. Such activities are known as crosscutting concerns that are difficult to modularize using existing activity modeling constructs. This paper presents an aspect-oriented approach to supporting separation of crosscutting concerns in activity models. An extension to activity modeling is introduced for encapsulating crosscutting activities in well-modularized aspects, which are in turn composed with base activities by a specialized aspect weaver in a systematic way.

Alarm correlation for fault management in large telecommunication networks demands scalable and reliable algorithms. In this paper, we propose a clustering based alarm correlation approach using sequential proximity between alarms. We define two novel distance metrics appropriate for measuring similarity between alarm sequences obtained from interval-based division: 1) the two-digit binary metric that values the occurrences of two alarms in neighboring intervals to tolerate the false separation of alarms due to interval-based alarm sequence division, and 2) the sequential ordering-based distance metric that considers the time of arrival for different alarms within the same interval. We validate both metrics by applying them with hierarchical clustering using real-world cellular network alarm data. The efficacy of the proposed sequential proximity based alarm clustering is demonstrated through a comparative study with existing similarity metrics.

Biologically inspired soft computing paradigms such as neural networks are popular learning models adopted in online adaptive systems for their ability to cope with the demands of a changing environment. However, continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure the reliable performance of such systems. In this paper, we discuss a dynamic approach to validate the adaptive system component. Our approach consists of two run-time techniques: (1) a statistical learning tool that detects unforeseen data; and (2) a reliability measure of the neural network output after it accommodates the environmental changes. A case study on NASA F-15 flight control system demonstrates that our techniques effectively detect unusual events and provide validation inferences in a real-time manner.

The appeal of adaptive control to the aerospace domain should be attributed to the neural network models adopted in online adaptive systems for their ability to cope with the demands of a changing environment. However, continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure the reliable performance of such systems. In this paper, we present several advanced methods proposed for verification and validation (V&V) of adaptive control systems, including Lyapunov analysis, statistical inference, and comparison to the well-known Kalman filters. We also discuss two monitoring tools for two types of neural networks employed in the NASA F-15 flight control system as adaptive learners: the confidence tool for the outputs of a Sigma-Pi network, and the validity index for the output of a Dynamic Cell Structure (DCS) network.

As a special type of Self-Organizing Maps (SOM), the Dynamic Cell Structures (DCS) network has topology-preserving adaptive learning capabilities that can, in theory; respond and learn to abstract from a wide variety of complex data manifolds. However, the highly complex learning algorithm and non-linearity behind the dynamic learning pose serious challenge to validating the performance of DCS and impede its spread in control applications; safety-critical systems in particular. In this paper, we analyze the performance of DCS network by providing sensitivity analysis on its structure and confidence measures on its predictions. We evaluate how the quality of each parameter of the network (e.g., weight) influences the output of the network by defining a metric for parameter sensitivity for DCS network. We present the validity index (VI), an estimated confidence associated with each DCS output, as a reliability-like measure of the network's prediction performance. Our experiments using artificial data and a case study oil a flight control application demonstrate that our analysis effectively measures the network performance and provides validation inferences in a real-time manner.

Biologically inspired soft computing paradigms such as neural networks are popular learning models adopted in adaptive control systems for their ability to cope with a changing environment. However, continual changes induce uncertainty that limits the applicability of conventional validation techniques to assure a reliable system performance.In this paper, we present a dynamic approach to estimate the performance of two types of neural networks employed in an adaptive flight controller: the validity index for the outputs of a Dynamic Cell Structure (DCS) network and confidence levels for the outputs of a Sigma-Pi (or MLP) network. Both tools provide statistical inference of the neural network predictions and an estimate of the current performance of the network. We further evaluate how the quality of each parameter of the network (e.g., weight) influences the output of the network by defining a metric for parameter sensitivity and parameter confidence for DCS and Sigma-Pi networks. Experimental results on the NASA F-15 flight control system demonstrate that our techniques effectively evaluate the network performance and provide validation inferences in a real-time manner.

The need for reliable identification and authentication is driving the increased use of biometric devices and systems. Verification and validation techniques applicable to these systems are rather immature and ad hoc, yet the consequences of the wide deployment of biometric systems could be significant. In this paper we discuss an approach towards validation and reliability estimation of a fingerprint registration software. Our validation approach includes the following three steps: (a) the validation of the source code with respect to the system requirements specification; (b) the validation of the optimization algorithm, which is in the core of the registration system; and (c) the automation of testing. Since the optimization algorithm is heuristic in nature, mathematical analysis and test results are used to estimate the reliability and perform failure analysis of the image registration module.

The appeal of biologically inspired soft computing systems such as neural networks in complex systems comes from their ability to cope with a changing environment. Unfortunately, adaptability induces uncertainty that limits the applicability of static analysis to such systems. This is particularly true for systems with multiple adaptive components or systems with multiple types of learning operation. This work builds a paradigm of dynamic analysis for a neuro-adaptive controller where different types of learning are to be employed for its online neural networks. We use support vector data description as the novelty detector to detect unforeseen patterns that may cause abrupt system functionality changes. It differentiates transients from failures based on the duration and degree of novelties. Further, for incremental learning, we utilize Lyapunov functions to assess real-time performance of the online neural networks. For quasionline learning, we define a confidence measure, the validity index, to be associated with each network output. Our study on the NASA F-15 Intelligent Flight Control System demonstrates that our novelty detection tool effectively filters out transients and detects failures; and our light-weight monitoring techniques supply sufficient evidence for an insightful validation. (c) 2006 Elsevier Inc. All rights reserved.

As a special type of Self-Organizing Maps, the Dynamic Cell Structures (DCS) network has topology-preserving adaptive learning capabilities that can, in theory, respond and learn to abstract from a much wider variety of complex data manifolds. However, the highly complex learning algorithm and non-linearity behind the dynamic learning pattern pose serious challenge to validating the prediction performance of DCS and impede its spread in control applications, safety-critical systems in particular. In this paper, we improve the performance of DCS networks by providing confidence measures on DCS predictions. We present the validity index, an estimated confidence interval associated with each DCS output, as a reliability-like measure of the network's prediction performance. Our experiments using artificial data and a case study on a flight control application demonstrate an effective validation scheme of DCS networks to achieve better prediction performance with quantified confidence measures.

The appeal of including biologically inspired soft computing systems such as neural networks in complex computational systems is in their ability to cope with a changing environment. Unfortunately, continual changes induce uncertainty that limits the applicability of conventional verification and validation (V&V) techniques to assure the reliable performance of such systems. At the system input layer, novel data may cause unstable learning behavior which may contribute to system failures. Thus, the changes at the input layer must be observed, diagnosed, accommodated and well understood prior to system deployment. Moreover, at the system output layer, the uncertainties/novelties existing in the neural network predictions also need to be well analyzed and detected during system operation. Our research tackles the novelty detection problem at both layers using two different methods. We use a statistical learning tool, Support Vector Data Description (SVDD), as a one-class classifier to examine the data entering the adaptive component and detect unforeseen patterns that may cause abrupt system functionality changes. At the output layer, we define a reliability-like measure, the validity index. The validity index reflects the degree of novelty associated with each output and thus can be used to perform system validity checks. Simulations demonstrate that both techniques effectively detect unusual events and provide validation inferences in a near-real time manner.

The appeal of including adaptive components in complex computational systems, such as flight control, is in their ability to cope with a changing environment. Continual changes induce uncertainty that limits the applicability of conventional verification and validation (V&V) techniques. In safety-critical applications, the mechanisms of change must be observed, diagnosed, accommodated and well understood prior to deployment. We present a nonconventional V&V approach suitable for online adaptive systems. We applied this approach to an adaptive flight control system that employs neural network learning for online adaptation. Presented methodology consists of a Novelty Detection technique and Online Stability Monitoring tools. The Novelty Detection technique is based on support vector data description that detects novel (abnormal) data patterns. The Online Stability Monitoring tools based on Lyapunov's stability theory detect unstable learning behavior in neural networks.

Rigorous Verification and Validation (V& V) techniques are essential for high assurance systems. Lately, the performance of some of these systems is enhanced by embedded adaptive components in order to cope with environmental changes. Although the ability of adapting is appealing, it actually poses a problem in terms of V&V. Since uncertainties induced by environmental changes have a significant impact on system behavior, the applicability of conventional V& V techniques is limited. In safety-critical applications such as flight control system, the mechanisms of change must be observed, diagnosed, accommodated and well understood prior to deployment. In this paper, we propose a non-conventional V&V approach suitable for online adaptive systems. We apply our approach to an intelligent flight control system that employs a particular type of Neural Networks (NN) as the adaptive learning paradigm. Presented methodology consists of a novelty detection technique and online stability monitoring tools. The novelty detection technique is based on Support Vector Data Description that detects novel (abnormal) data patterns. The Online Stability Monitoring tools based on Lyapunov's Stability Theory detect unstable learning behavior in neural networks. Cases studies based on a high fidelity simulator of NASA's Intelligent Flight Control System demonstrate a successful application of the presented V&V methodology.

One of the goals of verification and validation (V&V) activities for online adaptive control systems is providing assurance that they are able to detect novel system behaviors and provide adequate (safe) control actions. Novel (or abnormal) system behaviors cannot be enumerated or fully and explicitly described in requirements documentation. Therefore, they have to be observed and recognized during the operation. Novelty detection methods, therefore, provide an adequate approach for the V&V purposes. We propose a novelty detection method based on support sector data description (SVDD) as a candidate approach for validating adaptive control systems. As a one-class classifier, the support vector data description is able to form a decision boundary around the learned data domain with very little or no knowledge of data points outside the boundary (outliers). We apply the SVDD techniques for novelty detection as part of the validation on an intelligent flight control system (IFCS). Experimental results show that the SVDD can be adopted as an effective tool for finding indications of the safe region for the learned domain, whereby we are able to separate faulty behavior from normal events.