Network data is commonly available across various domains, sparking a surge in research dedicated to modeling and monitoring network systems. In the realm of network analysis with node attributes, the majority of existing studies utilize generalized linear models (GLMs) to establish connections between network topology and node attributes. However, these studies often overlook the incongruity between directional edges and directionless attributes within the context of directional networks, as well as the inadequacy of using only observable attributes to explain the network topology. In this paper, we introduce a novel Hurdle regression model with latent variables (HRML), which assigns four latent variables to each node to govern the directionality of interactions. By integrating observable attributes, our proposed model adeptly manages directional, sparse, and attributed networks. We further develop the HRML into its dynamic version (D-HRML) within the state space model framework to capture the temporal dynamics of network streams. An extended Kalman filter (EKF) is employed for optimal parameter estimation. Ultimately, we devise a monitoring scheme based on the generalized likelihood ratio test (GLRT) to detect abrupt changes across diverse scenarios. Extensive simulations demonstrate that our proposed method outperforms several competitive approaches, particularly in detecting shifts in interaction propensities. A case study utilizing the Enron E-mail corpus further substantiates the high efficiency of our methodology.

ABSTRACT Video anomaly detection (VAD) has the capability to identify product surface defects and has been used in some real industrial applications. Given the typically unlabeled nature of industrial data, most existing research employs unsupervised learning to detect surface defects. However, existing approaches, such as those based on tensor decomposition, assume a low‐rank video background and are only valid when the objects being inspected are static. If the objects are in motion, such as on a conveyor belt, which is common in modern manufacturing lines, these methods become ineffective. Additionally, deep learning‐based video prediction methods often suffer from error accumulation. In this work, a novel method called iterative prediction and decomposition (IPAD) has been proposed to address these challenges. In each iteration, the method initially predicts the subsequent frame based on information from the previous frames. Then, sparse surface defects are separated from the defect‐free regions through the application of an optimization algorithm. The proposed method relaxes the low‐rank assumption, making it applicable to non‐low‐rank backgrounds, and utilizes previously predicted frames to mitigate error accumulation. The prediction network is fine‐tuned using videos generated from industrial images, and its performance is evaluated by datasets with real factory environments and simulated anomalies. The IPAD method demonstrates superior performance in predicting defect‐free parts and detecting sparse surface defects.

Complex manufacturing systems usually contain a large number of variables. Dominated by certain engineering mechanisms, these variables show complicated relationships that cannot be effectively expressed by simple correlation matrices or functions, thus increasing the difficulty of modeling and monitoring these systems. The directed graphical model (DGM) has been used as a flexible tool for describing the relationship among variables in complex systems. However, the DGM treats all variables equally and fails to consider the structural information among them that usually exists. To address this problem, an extended directed graphical model (EDGM) and related parameter estimation, monitoring, and structure learning methods are proposed in this work. Taking prior engineering knowledge into consideration, the EDGM assigns variables into groups and uses groups of variables as nodes in the graph model. By adding hidden state variables to each node, the EDGM can effectively represent the relationship within and between nodes and provide promising monitoring performance. Numerical experiments and a real-world case study of the monocrystalline silicon growth process are performed to verify the effectiveness of the proposed methods.

The operating process of complex systems usually manifest in multiple distinct operating modes. In the case of a wind turbine, for example, its operating mode is highly influenced by the wind condition, which changes dynamically in natural environment. The SCADA system plays a crucial role in collecting various parameters from wind turbines, facilitating the differentiation, and modeling of distinct operating modes. However, the challenge lies in the excessive dimensionality of variables in SCADA data, making modeling efforts both intricate and inefficient. In this study, we leverage the engineering knowledge on the hierarchical structure of the variables in wind turbine, and propose a novel method to efficiently cluster the data temporally by operating modes. Our methodology involves initially clustering variables according to subsystems and implementing temporal clustering within each subsystem. Subsequently, we introduce a novel graph neural network to extract and concatenate features from all subsystems, enabling the discrimination of the operational mode of the entire system. Finally, we model these features to make predictions of the output power, and the prediction residual can be used for monitoring. Performance evaluations on both numerical experiments and real-world wind turbine datasets attest to the effectiveness and superiority of the proposed methods.

Multimode processes are common in modern industry and refer to processes that work in multiple operating modes. Motivated by the torque control process of a wind turbine, we determine that there exist two types of changes in multimode processes: (i) mode transitions and (ii) parameter changes. Detecting both types of changes is an important issue in practice, but existing methods mainly consider one type of change, and thus, do not work well. To address this issue, we propose a novel modeling framework for the offline change point detection problem of multimode processes, which simultaneously considers mode transitions and parameter changes. We characterize each mode with a parametric cost function and formulate the problem as an optimization model. In the model, two penalty terms penalize the number of change points, and a series of constraints specify the multimode characteristics. With certain assumptions, the asymptotic property ensures the accuracy of the model solution. To solve the model, we propose an iterative algorithm and develop a multimode-pruned exact linear time (multi-PELT) method for initialization. The simulation study and the real case study demonstrate the effectiveness of our method against the state-of-the-art methods in terms of the accuracy of change point detection, mode identification, and parameter estimation.

As data with network structures are widely seen in diverse applications, the modeling and monitoring of network data have drawn considerable attention in recent years. When individuals in a network have multiple types of interactions, a multilayer network model should be considered to better characterize its behavior. Most existing network models have concentrated on characterizing the topological structure among individuals, and important attributes of individuals are largely disregarded in existing works. In this article, first, we propose a unified static Network Generative Model (static-NGM), which incorporates individual attributes in network topology modeling. The proposed model can be utilized for a general multilayer network with weighted and directed edges. A variational expectation maximization algorithm is developed to estimate model parameters. Second, to characterize the time-dependent property of a network sequence and perform network monitoring, we extend the static-NGM model to a sequential version, namely, the sequential-NGM model, with the Markov assumption. Last, a sequential-NGM chart is developed to detect shifts and identify root causes of shifts in a network sequence. Extensive simulation experiments show that considering attributes improves the parameter estimation accuracy and that the proposed monitoring method also outperforms the three competitive approaches, static-NGM chart, score test-based chart (ST chart) and Bayes factor-based chart (BF chart), in both shift detection and root cause diagnosis. We also perform a case study with Enron E-mail data; the results further validate the proposed method.

Acknowledging the significance of information propagation and individual adaptive behavior has been regarded as an indispensable prerequisite for a complete understanding of epidemic spreading. Recent studies have widely considered the metapopulation model, where epidemics spread over a single layer of physical networks via individual mobility. However, these advances neglected the interventions of accompanied information and individual behavior response related to epidemics. In this article, we develop a coupled epidemic-information propagation model on multiplex metapopulation networks leveraging the microscopic Markov chain (MMC) approach, aiming to explore the spatiotemporal characteristics of epidemic spreading process. Taking the individual adaptive behavior into account, the stranding mechanism based on infection level and medical resources is introduced to capture the population size dynamics during individual mobility among different patches. Theoretical epidemic threshold is analytically derived under the improved framework. Extensive numerical simulations are performed to validate our theoretical analysis and further examine the impacts of information propagation and spreading parameters on epidemic threshold and steady-state prevalence. Our results indicate that both the scale of information diffusion and the specific configuration of spreading parameters can significantly suppress the epidemic prevalence. These findings shed a novel light on theoretical research and decision-making of coupled epidemic-information process in the spatiotemporal perspective.

Wind power forecasting is critical for ensuring the reliability of wind power systems. A wind turbine consists of several subsystems, each containing various sensors that collect multivariate time series. These subsystems can naturally classify the turbines into clusters. This clustering strongly correlates variables within the same cluster, and the correlation between two clusters can be derived from engineering knowledge. In this study, we propose a hierarchical multivariate time series forecasting method based on a spatial-temporal graph convolution network (HMTGCN) to forecast wind power by leveraging engineering knowledge. The model uses a spatial-temporal graph neural network (GNN) containing a graph learning module to extract features from each time-series cluster. These features are concatenated to form graph data at the cluster level, which are subsequently processed by a graph convolution network. We evaluated the performance using simulation experiments and a real-life wind power dataset, and the results showed that the proposed method improved the prediction performance by 8.99% on average, which demonstrated the effectiveness and superiority of our approach.

Online change detection of multimode processes is important for process monitoring and control, which aims to timely and accurately detect two types of changes: 1) mode changes and 2) parameter changes. However, the existing online methods mainly focus on one type of change and, thus, have difficulty capturing the complex change structure. Motivated by this, we propose a novel Bayesian online change detection method for multimode processes (BCD-MMP). Specifically, the complex change structure is characterized by three state variables (the latest change point (LCP), the mode of the current segment, and the indicator variable of parameter changes). When a new data point arrives, we infer the posterior distribution of the state variables. To make the inference process tractable, we develop a sampling-based inference algorithm, and a pruning strategy is also provided to improve the computation efficiency. Simulation and real case studies of wind turbine torque control process indicate that the proposed method can achieve better online change detection performance than the state-of-the-art methods.

In complex manufacturing systems, materials are processed by machines sequentially before final products are obtained, forming a Multistage Manufacturing Process (MMP). In a modern massive production factory, it is common that multiple MMPs work simultaneously to fulfill productivity needs. Multiple MMPs may contain machines with different running times, processing the same types of products, but with different specifications. In such a Multiple MMP system, it is critical for machines to learn from each other to gain optimal parameter settings of each. However, traditional machine learning methods usually fail to consider both similarities and unique features among multiple processes at the same time. To address this problem, a partial domain generalization method is proposed based on the thought of transfer learning to combine useful information from historical processes while maintaining features from the in-production process. The proposed method also suits for problems with the assumption that only partial input variables are available. Studies reveal that the proposed method has superior prediction performance over traditional machine learning methods and some widely-used transfer learning methods.

Analysis of online reviews has attracted great attention with broad applications. Often times, the textual reviews are coupled with the numerical ratings in the data. In this work, we propose a probabilistic model to accommodate both textual reviews and overall ratings with consideration of their intrinsic connection for a joint sentiment-topic prediction. The key of the proposed method is to develop a unified generative model where the topic modeling is constructed based on review texts and the sentiment prediction is obtained by combining review texts and overall ratings. The inference of model parameters are obtained by an efficient Gibbs sampling procedure. The proposed method can enhance the prediction accuracy of review data and achieve an effective detection of interpretable topics and sentiments. The merits of the proposed method are elaborated by the case study from Amazon datasets and simulation studies.

Functional directed graph model (DGM) learning has been widely used to analyze complex systems. Most existing works assume that the functional signals are complete, which in reality is not true. To address this problem, in this study, a framework for DGM learning with incomplete signals is proposed. Specifically, a penalty term that integrates information from the graph structure is added to the Maximum Margin Matrix Factorization (MMMF) objective function. The proposed method can be used with a known structure to estimate the functional relationship between nodes or with an unknown structure to estimate the relationship together with the graph structure. Numerical experiments and a real-world case study of monocrystalline silicone manufacturing are performed to verify the effectiveness of the proposed method when the signal matrices are incomplete.

Nowadays advanced sensing technology enables real-time data collection of key variables during manufacturing, known as multi-channel profiles. These data facilitate in-process monitoring and anomaly detection, which have been extensively studied in recent years. However, most studies treat each profile as a whole, e.g., a high-dimensional vector or function, and construct monitoring schemes accordingly. As a result, these methods cannot be implemented until the entire profile has been obtained, leading to long detection delay especially if anomalies occur in early sensing points of the profile. In addition, they require that profiles of different samples have the same time length and feature location, yet additional time-warping operation for real misaligned samples may weaken the anomaly patterns. To address these problems, this article proposes an in-profile monitoring (INPOM) control chart, which not only gives the feasibility of detecting anomalies inside the profile, but also can handle the misalignment problem of different samples. In particular, our INPOM scheme is built upon state space model (SSM). To better describe the clustered between-profile correlation and avoid overfitting, SSM is extended to a regularized SSM (RSSM), where regularizations are imposed as prior information and expectation maximization algorithm is integrated for posterior maximization to efficiently learn the model parameters. Furthermore, a monitoring statistic based on one-step-ahead prediction error of RSSM is constructed for INPOM control chart. Thorough numerical studies and real case studies demonstrate the effectiveness and applicability of our proposed RSSM-INPOM framework.

Human life is deeply influenced by infectious diseases. A vaccine, when available, is one of the most effective ways of controlling the spread of an epidemic. However, vaccine shortage and uncertain vaccine effectiveness in the early stage of vaccine production make vaccine allocation a critical issue. To tackle this issue, we propose a multi-objective framework to optimize the vaccine allocation strategy among different age groups during an epidemic under vaccine shortage in this study. Minimizing total disease onsets and total severe cases are the two objectives of this vaccine allocation optimization problem, and the multistage feature of vaccine allocation are considered in the framework. An improved Strength Pareto Evolutionary Algorithm (SPEA2) is used to solve the optimization problem. To evaluate the two objectives under different strategies, a deterministic age-stratified extended SEIR model is developed. In the proposed framework, different combinations of vaccine effectiveness and vaccine production capacity are investigated, and it is identified that for COVID-19 the optimal strategy is highly related to vaccine-related parameters. When the vaccine effectiveness is low, allocating most of vaccines to 0-19 age group or 65+ age group is a better choice under a low production capacity, while allocating most of vaccines to 20-49 age group or 50-64 age group is a better choice under a relatively high production capacity. When the vaccine effectiveness is high, a better strategy is to allocate vaccines to 65+ age group under a low production capacity, while to allocate vaccines to 20-49 age group under a relatively high production capacity.

User-generated content including both review texts and user ratings provides important information regarding the customer-perceived quality of online products and services. This article proposes a modeling and monitoring method for online user-generated content. A unified generative model is constructed to combine words and ratings in customer reviews based on their latent sentiment and topic assignments, and a two-chart scheme is proposed for detecting shifts of customer responses in dimensions of sentiments and topics, respectively. The proposed method shows superior performance in shift detection, especially for the sentiment shifts in customer responses, based on the results of simulation and a case study.

Profile data have been widely used in quality control practice. However, in some applications, incomplete profiles, or profiles with continuous missing values, are frequently encountered. Imputation of continuous missing values in profiles for quality control is a challenging problem. In this work, we propose a latent feature model and a two‐step learning algorithm to reconstruct missing values. In the model, an unsupervised deep‐learning technique, variational autoencoder (VAE), is utilized to learn the latent features from profile data. Then a supervised variational coding (SVC) scheme is proposed to map the incomplete profile data to the learned latent features, and imputation is then made based on the latent features. The performance of the proposed method is evaluated by simulated and real datasets, and the results prove the effectiveness and superiority of the proposed method.

Quality prediction for small-batch production processes is a complex problem due to limitations in available training samples. In this study, a multisource domain adaptation joint-Y partial least square (PLS) method is proposed to learn the similarities between domains and use them to construct a quality prediction model. Without constraints on the number of source and target domains, the proposed method can transfer more historical information for the in-operation process than traditional methods. Numerical experiments and a real-world case study of quality prediction in computer wafer production are performed to verify the effectiveness of the proposed method. The results show that the prediction accuracy of the proposed method is high in cases with few training samples in the target domain compared to the accuracies of the joint-Y PLS model and the traditional PLS model.

Interaction event networks, which consist of interaction events among a set of individuals, exist in many areas from social, biological to financial applications. The individuals on networks interact with each other for several possible reasons, such as periodic contact or reply to former interactions. Regarding these interaction events as expectations based on previous interactions is crucial for understanding the underlying network and the corresponding dynamics. Usually, any change on individuals of the network will reflect on the pattern of their interaction events. However, the causes and expressed patterns for interaction events on networks have not been properly considered in network models. This article proposes a dynamic model for interaction event networks based on the temporal point process, which aims to incorporate the impact from historical interaction events on later interaction events considering both network structure and node connections. A network representation learning method is developed to learn the interaction event processes. The proposed interaction event network model also provides a convenient representation of the rate of interaction events for any pair of sender-receiver nodes on the network and therefore facilitates monitoring such event networks by summarizing these pairwise rates. Both simulation experiments and experiments on real-world data validate the effectiveness of the proposed model and the corresponding network representation learning algorithm.

With the development of sensor and communication technology, condition-based maintenance (CBM) attracts increasing attention, especially for multi-component systems. This paper aims to investigate the optimal CBM policy under periodic inspection for a K-out-of-N: G system, where economic dependency, stochastic dependency and imperfect maintenance are emphasized. The objective is to minimize the expected long-run discounted cost. In the model, the cumulative degradation of each component is modeled by heterogeneous stochastic processes, the dependence among all components is characterized by a copula function, and the imperfect maintenance is represented by a reduction in the degradation level. Since the system has Markov property, we solve the CBM optimization problem based on Markov decision process (MDP) framework. To ease the computation burden, we discretize the continuous state space and then use the value iteration algorithm with Monte Carlo simulation to find the optimal inspection interval and the optimal CBM policy. Numerical studies for a 1-out-of-2: G system are conducted to systematically examine the impacts of degradation processes, copula functions and imperfect maintenance on the optimal maintenance decisions, which provides insights for multi-component system maintenance. A sensitivity analysis of cost-related parameters is also performed.

Anomaly detection has been extensively studied over the past decades; however, there are still various challenges due to the complex structures of the real-world datasets. First, only a few methods in the literature provide insight into the datasets that have both categorical and continuous attributes, and even fewer of them are sensitive to the dependencies between the two types of attributes. Second, a real-world dataset tends to be more complex in its structure, and the categorical attributes are usually hierarchically correlated, which has been largely ignored by the existing outlier detection approaches. Following this line of reasoning, we propose a distributed outlier detection method for mixed attribute datasets, especially with hierarchical categorical attributes. The proposed method accounts for the dependencies between categorical and continuous attributes rather than treating them as two separate parts. In addition, the proposed method is able to capture the hierarchical structure among categorical attributes. The experimental results on a real-world dataset and a simulation study show its superior performance in terms of both the detection accuracy and time efficiency.