Alarm floods can overload control room operators during abnormal operation of process plants. Online ”alarm floodclassification” (AFC) assigns an unfolding alarm-event stream to a predefined alarm flood class while only a prefix of theepisode is observable. In deployment, the observed prefix can deviate from curated historical logs due to two distinctuncertainty sources: (i) alarm-log degradations within extracted episodes, including missing events, spurious events,and timing uncertainty that can alter recovered event ordering; and (ii) pipeline-induced uncertainty, including delayedalarm flood detection. We propose AFC-RobustBench, a perturbation-based statistical robustness benchmarkingprotocol for prefix-based online AFC that models both these sources as perturbation functions acting on alarm-eventstreams and supports reproducible mixed-perturbation composition. The method-agnostic protocol reports degradationprofiles over severity and observation progress, scalar robustness scores, and uncertainty estimates across Monte-Carlodraws and repeated folds. Applying this benchmarking protocol, we benchmarked seven relevant AFC methods on twoprocess-industry datasets. Results indicate that nominal performance on clean test data can misrepresent behavior underdegraded alarm logs and detection delays, and that clean-performance rankings can differ substantially from robustnessrankings under online, prefix-based observation. These findings demonstrate the usefulness of AFC-RobustBench forassessing AFC methods under realistic degraded and delayed observation conditions.
Cyber-physical system (CPS) forecasting models depend on sensor streams with noisy, biased, missing, or temporally misaligned readings, yet standard forecasting evaluation often selects models by nominal error without showing whether they remain robust under such faults. We introduce SensorFault-Bench, a shared CPS-grounded sensor-fault stress-test protocol for evaluating forecasting architectures and robustness-improvement methods, and an operational taxonomy organizing the method comparison. Across four real-world datasets and eight scored scenarios governed by a standardized severity model, it reports worst-scenario degradation, clean mean squared error (MSE), and worst-scenario fault-time MSE, separating relative robustness from absolute error. A disjoint fault-transfer split lets explicit fault-training methods train on adjacent fault families while evaluation uses separate benchmark scenarios. Empirically, forecasting architectures favored by clean MSE can degrade sharply under faults, and clean-MSE rankings can disagree with worst-scenario fault-time error rankings. Chronos-2, the evaluated zero-shot foundation-model representative, matches or trails the last-value naive forecaster in clean MSE on the two single-target datasets and has the largest worst-scenario degradation on ETTh1 and Traffic, where all channels are forecast targets. For the evaluated robustness-improvement method set, paired deltas show selective degradation reductions: projected gradient descent adversarial training and randomized training lead where value faults dominate observed degradation, while fault augmentation leads where availability faults dominate. SensorFault-Bench provides open-source code, documented data access, and reproduction and extension guides, so new datasets, architectures, and robustness-improvement methods can be evaluated under the same CPS sensor-fault robustness protocol.
Alarm floods remain a challenge in industrial operations, potentially overwhelming human operators with excessive alarm notifications during abnormal situations. To address this, alarm flood classification (AFC) methods utilize historical data to classify recurring alarm patterns automatically. However, the practical utility of these methods may be limited by potential degradation in alarm data quality, resulting from sensor faults, communication errors, or detection delays, which can substantially compromise their classification accuracy and reliability. This paper proposes a novel methodology to systematically analyze the robustness of AFC methods against realistic alarm data quality issues. We introduce four distinct perturbations: missing alarms, false alarms, delayed alarm flood detection, and alarm reordering, to replicate real-world alarm data degradation. We evaluate our methodology using a novel alarm dataset derived from the Tennessee-Eastman process, while examining the robustness of six relevant AFC methods from the literature. The results demonstrate significant variations in robustness across different AFC methods and perturbation types, providing insights into their practical reliability under various realistic scenarios.
Supervised machine learning has presented impressive results in the industrial domain - when it can depend on reliable ground truth labels. However, the concept of golden labels often represents only an idealization of reality. Instead, data labeling can be highly subjective and hence susceptible to corruption, such as unintentional errors, resulting in socalled label noise. Since label noise can significantly impair the performance of supervised machine learning models, it can be helpful to assess their robustness to label noise when evaluating existing limitations. Despite the large number of publications offering best practices and techniques for making models more robust to label noise, literature lacks generic tools that would allow data scientists and industrial practitioners to test and compare which technique works best for their given model. To address this gap, we propose a model-agnostic, reusable framework for measuring the robustness of supervised machine learning models to label noise.
Supervised machine learning (ML) has achieved significant outcomes in the industrial domain, dependent on the availability and accuracy of ground truth labels. However, the usual assumption of a ground truth to exist in the data often represents only an idealization of real-world conditions, as data labeling can be subjective and prone to errors, leading to so-called label noise. Such noise can significantly degrade model performance. Although extensive research has identified methods to improve model robustness against label noise, there is a notable absence of generic, reusable frameworks that allow industrial practitioners to systematically assess model robustness. To address this gap, we extend our previous work and propose a model-agnostic framework designed specifically for evaluating robustness against label noise. Our framework incorporates two distinct label noise perturbation mechanisms: an instance-independent symmetric perturber and an instance-dependent one. We demonstrate the utility of our extended framework through empirical evaluations on two industrial datasets using six relevant time series classification methods from the literature. The results highlight the significant vulnerability of supervised ML models to both noise types and underscore the value of our framework in uncovering these robustness limitations.
Being able to pinpoint the type of anomaly goes beyond basic anomaly detection, which simply flags unusual events. It empowers users to gain a deeper understanding of the problem. However, most recognition/classification algorithms struggle with unknown data (Open Set Recognition (OSR) problem), which poses a critical limitation for industrial applications where unforeseen faults can occur. This paper investigates Siamese Neural Networks (SNNs) for recognizing anomalies in industrial time series data. Our use case involves developing an anomaly detection system for power plant operators. We evaluate the effectiveness of SNN s using real data from a power plant in Germany, and two additional public datasets (MaFaulDa and TEP), demonstrating SNN s as a transferable solution for overcoming the OSR problem in the industrial domain.
The escalating global emissions attributable to commercial buildings call for the integration of digital technologies to enhance energy efficiency and occupant comfort. Cyber-physical control systems (CPCS) can collect data from various sensors and actuators in the building to provide valuable insights to building owners. However, the unstructured metadata text fields in these systems pose a challenge in leveraging artificial intelligence and machine learning solutions for building management. This paper proposes a solution using pre-trained large language models for building metadata normalization that addresses inconsistencies in the point text metadata across buildings. The evaluation of this solution is performed using two publicly-available CPCS datasets, revealing its ability to structure the unstructured natural language metadata without fine tuning or training from scratch.
Despite the high degree of automation in industrial control systems, human operators in industrial plants play a critical role in ensuring uptime, production quality, and safety. Plant operators do so by not only monitoring the process but also intervening when the process runs into unusual or exception situations. In this paper, we present to ensure smooth plant operation by automatically identifying and investigating potential upcoming issues as well as providing recommendations to plant operators on how to address them with confidence. This is achieved by applying Artificial Intelligence (AI) techniques including deep learning, process mining, and graph search, on historical industrial process data such as alarm and event data, audit trails, engineering documents, and safety procedures. Our solution has been validated on data from the Draugen oil field operated by the Norwegian oil and gas company, OKEA.
Reliable functionality in anomaly detection in thermal image datasets is crucial for defect detection of industrial products. Nevertheless, achieving reliable functionality is challenging, especially when datasets are image sequences captured during equipment runtime with a smooth transition from healthy to defective images. This causes contamination of healthy training data with defective samples. Anomaly detection methods based on autoencoders are susceptible to a slight violation of a clean training dataset and lead to challenging threshold determination for sample classification. This paper indicates that combining anomaly scores leads to better threshold determination that effectively separates healthy and defective data. Our research results show that our approach helps to overcome these challenges. The autoencoder models in our research are trained with healthy images optimizing two loss functions: mean squared error (MSE) and structural similarity index measure (SSIM). Anomaly score outputs are used for classification. Three anomaly scores are applied: MSE, SSIM, and kernel density estimation (KDE). The proposed method is trained and tested on the 32 × 32-sized thermal images, including one contaminated dataset. The model achieved the following average accuracies across the datasets: MSE, 95.33%; SSIM, 88.37%; and KDE, 92.81%. Using a combination of anomaly scores could assist in solving a low classification accuracy. The use of KDE improves performance when healthy training data are contaminated. The MSE+ and SSIM+ methods, as well as two parameters to control quantitative anomaly localization using SSIM, are introduced.
Industrial ML models are primarily data-driven. Therefore, one of the main focus for monitoring the model should be towards identifying the drifts in the data that might affect the performance of the model. The traditional drift detecting methods are usually based on some assumptions related to the underlying data such as no inter-dependence. However industrial sensor data typically consists of time series data, which is collected at regular intervals. Therefore, detecting drift in dependent data where the current readings depend on the previously registered readings demands a different approach. Existing solutions require either the ground truth, a fixed size, or the underlying model details. We propose an Unsupervised Drift Detection method for industrial Time series data or UDDT, a generic approach with no such pre-requisites. In our approach, we can check whether two series belong to the same model. Apart from detecting the drift in the two series, it can also provide the rationale behind the observed drift, i.e., whether the drift is due to a difference in stationarity, correlation structures, or noise distributions. We evaluate the UDDT on two datasets to demonstrate its correctness and the trust regions under various circumstances.
The performance of machine learning models can be significantly impacted by variations in data quality. Typically, conventional model testing does not examine how robust the model would be in the face of potential data quality deterioration. In an industrial use case, however, data quality is a pertinent issue, as sensors are susceptible to a variety of technical and external issues that may result in poor data quality over time. In order to develop robust machine learning models, industrial data scientists must understand the sensitivity of their models against data quality issues, through the application of an appropriate and comprehensive testing solution. In this work, we propose a generic framework for systematically analyzing the impact of data quality issues on the performance of machine learning models by intentionally applying gradual perturbations to the original time series data. The evaluation is performed using a benchmark industrial process consisting of multivariate time series from sensors in a complex chemical process.
Alarm flood similarity analysis (AFSA) methods group similar historical alarm floods and serve as a preprocessing step for further analysis. The discovered alarm flood clusters can be used for online classification to alert the operator if a similar situation recurs. In state-of-the-art AFSA methods, however, similarity measures are calculated based on only a small set of dynamic properties. As a result, more complex dynamic similarities between alarm floods are left out. To address and solve this limitation, a novel machine learning-based AFSA method is presented in this paper that uses alarm series as input to a recently proposed multivariate time series transformation method called “minimally random convolutional kernel transform with multiple pooling operators and transformations” (MultiRocket). This method is used to extract a variety of features and considers the relationships between different alarms, their dynamic properties, and the global structure of an alarm flood to a greater extent. Using an openly accessible dataset based on the simulated “Tennessee-Eastman” process, our method is compared with four relevant methods from the literature. Our results show that placing a greater emphasis on dynamics and structures improves the proposed AFSA’s overall performance and robustness in cases where higher-order alarm flood similarities are of interest.
Die jüngsten Fortschritte bei industriellen Prozesssteuerungssystemen haben zu großen Mengen an Produktionsdaten geführt, insbesondere zu Zeitreihendaten von verschiedenen Geräten in einer Prozessanlage. Diese Daten bieten die Möglichkeit für KI-basierteAnwendungsfälle, die Anlagenbetreibern helfen können, komplizierte Anlagenszenarien zu verstehen und vorherzusagen, damit sie ihre Anlagen effizienter und sicherer betreiben können. Ein wichtiger AI use case kann hierbei die Erkennung von Fehlern in Feldgeräten bieten, aufgrund des hohen Aufwands solche Fehler aufzuspühren und zu beheben. In diesem Beitrag wird die Erkennung von Fehlern in der verfahrenstechnischen Anlagenausrüstung mit Hilfe von neuronalen Netzwerkarchitekturen und statistischen maschinellen Lernmodellen behandelt.
The large amount of data generated by industrial plants provides an excellent opportunity to use Machine Learning (ML) for a better understanding of plant behaviour. Thus, supporting plants operators in running their plants efficiently. An example of an anomaly detection method is to help plant operators detect if their plant is still running normally or if curative actions are needed. Users of ML-based solutions often complain of a challenge called lack of interpretability, i.e., the degree to which one can understand the model's outcome. To address the need for better interpretability of ML models, the re-search field eXplainable Artificial Intelligence (XAI) has recently received increasing attention in the industrial domain. This paper performs a survey and investigates different XAI techniques that can be applied for detecting anomalies in industrial plant assets. The paper focuses on multivariate time-series data because a) it is the most predominant type of data in industrial systems and b) all the available XAI techniques have applications for various data types such as images, tabular, or textual data. However, these techniques are generally not well suited to explain anomalies in multivariate time-series. To solve this problem, we build an anomaly detection method for the multivariate time-series data generated by the industrial simulators using auto-encoders. Based on feature attribution, examples, and trees, seven different XAI techniques are ideated, developed, and discussed. Out of the seven XAI techniques, a SHAP based explainer, called DTFS, correctly identified the root cause of the anomaly with an accuracy of 86% and took 1.53 seconds to explain our benchmark system.
In process-based industries, modern process control systems have become data-driven and collect a vast amount of data from sensors in the field and alarm information. The collected data provides an opportunity for the data scientist to learn from historical data and apply Machine Learning (ML) models to automate the process control systems. Thus, assisting the plant operators in making informed decisions. In this paper, we focus on the alarm prediction of control systems. Alarm prediction assists plant operators in observing the functioning of plants and taking corrective measures beforehand to avoid upcoming failure situations. A data pipeline is proposed in this paper comprising two approaches for alarm prediction. Both the approaches consider the alarm log simulated data from an industrial three-phase separator process typically found in oil fields. The first approach requires domain knowledge regarding the alarm thresholds values, and ML models are trained using the threshold values to perform alarm prediction. The second approach comprises ML models trained independently of the alarm threshold values, thus providing the alarm prediction time window. The alarm prediction time window provides the plant operator sufficient time to act on an impending failure. As the outcome from the two approaches is different, Long short-term memory (LSTM) is the best performing model for the first approach with an RMSE value of 0.03. For the second approach, a fully convolutional network (FCN) is the best performing model for time windows of 20, 40, 60, and 120 minutes, and LSTM is the best performing model with an accuracy 94% for the time window of 10 minutes.
Condition monitoring of electrical equipment is used to detect faults in order to make repairs before an expensive or even catastrophic failure. For example, Infrared Thermography (IRT) cameras are able to detect hot spots indicative of such problems. Machine learning is a good tool to classify IRT images into healthy (no hot spot) and faulty (hot spot). In this paper, we explore the use of Siamese neural networks to compare black and white thresholded images derived from the original infrared images to detect hot spots. The network is trained on synthetic data. It is tested on experimental data derived from a low voltage switchgear panel. The algorithm shows good results on the experimental data, being able to detect all faults and not giving a false alarm on the healthy experiment.
Alarm flood similarity analysis (AFSA) methods group similar historical alarm floods and serve as a preprocessing step for further analysis. The discovered alarm flood clusters can be used for online classification to alert the operator if a similar situation recurs. In state-of-the-art AFSA methods, however, similarity measures are calculated based on only a small set of dynamic properties. As a result, more complex dynamic similarities between alarm floods are left out. To address and solve this limitation, a novel machine learning-based AFSA method is presented in this paper that uses alarm series as input to a recently proposed multivariate time series transformation method called “minimally random convolutional kernel transform with multiple pooling operators and transformations” (MultiRocket). This method is used to extract a variety of features and considers the relationships between different alarms, their dynamic properties, and the global structure of an alarm flood to a greater extent. Using an openly accessible dataset based on the simulated “Tennessee-Eastman” process, our method is compared with four relevant methods from the literature. Our results show that placing a greater emphasis on dynamics and structures improves the proposed AFSA’s overall performance and robustness in cases where higher-order alarm flood similarities are of interest.
The recent advances in industrial process control systems have led to vast amounts of production data, especially time-series data from various equipment in a process plant. This data provides an opportunity for AI-based use cases that can help plant operators to understand and predict complicated plant scenarios, allowing them to run their plants more efficiently and securely. Here, asset fault detection can provide a valuable AI use case, due to the time and cost-intensive process of discovering and rectifying from such faults. In this paper asset fault detection in industrial process plants is addressed with the help of neural network architectures and statistical machine learning models.
This paper reports the first results of a comparison of human and algorithmic anomaly detection. We are interested in how human and automated anomaly detection can be combined in the most beneficial way to improve how fault detection is practiced in building maintenance. Open source datasets with sensor data were annotated by persons with low subject matter experience, and compared with a Convolutional Autoencoder Neural Network (CAE) as well as a dedicated time series algorithm (DeepAnT), and detection metrics of human and algorithmic procedures are compared. Future comparisons will include higher levels of expertise on the human side, and more sophistication/training amount on the algorithm side. We close by discussing the advantages and caveats of our approach.
Driven by the increasing degree of automation, industrial production plants have become very data reliant, which poses a great potential for machine learning applications. AutoML is a fledgling research topic that has lately gained much attention in the industrial domain. However, existing applications of AutoML are limited, as industrial systems typically involve time-series data, while AutoML solutions for this type of data seem to be still underrepresented. On the contrary, existing AutoML libraries provide better solutions for, e.g., image, textual, tabular or categorical data. To close this gap to the data types and requirements that are typically found in the industrial domain, especially w.r.t. time-series data, a reusable framework is presented that provides native support for time-series models. The framework is equipped with 1) optimization support for a large number of model and hyperparameter configurations, 2) a warm starting module that performs meta-learning, 3) native support for time-series models, 4) an API for enabling user-defined custom models, and 5) a User Interface that provides a holistic view of the optimization results and deployment instructions. Experimental results show the framework’s competitive performance on time-series data and the effectiveness of the warm starting module in accelerating the optimization procedure. A qualitative analysis of the API is done to showcase the framework’s usability regarding defining custom models.
Martin Atzmüller合作论文数Department for Artificial Intelligence;and Applied Computer Science;University of W??rzburg2
Marco Gaertler合作论文数Universitat Karlsruhe (TH);Institute of Theoretical Computer Science2