In order to identify a system (module) embedded in a dynamic network, one has to formulate a multiple-input estimation problem that necessitates certain nodes to be measured and included as predictor inputs. However, some of these nodes may not be measurable in many practical cases due to sensor selection and placement issues. This may result in biased estimates of the target module. Furthermore, the identification problem associated with the multiple-input structure may require determining a large number of parameters that are not of particular interest to the experimenter, with increased computational complexity in large-sized networks. In this paper, we tackle these problems by using a data augmentation strategy that allows us to reconstruct the missing node measurements and increase the accuracy of the estimated target module. To this end, we develop a system identification method using regularized kernel-based methods coupled with approximate inference methods. Keeping a parametric model for the module of interest, we model the other modules as Gaussian Processes (GP) with a kernel given by the so-called stable spline kernel. An Empirical Bayes (EB) approach is used to estimate the parameters of the target module. The related optimization problem is solved using an Expectation-Maximization (EM) method, where we employ a Markov-chain Monte Carlo (MCMC) technique to reconstruct the unknown missing node information and the network dynamics. Numerical simulations on dynamic network examples illustrate the potentials of the developed method.

In order to identify one system (module) in an interconnected dynamic network, one typically has to solve a Multi-Input-Single-Output (MISO) identification problem that requires identification of all modules in the MISO setup. For application of a parametric identification method this would require estimating a large number of parameters, as well as an appropriate model order selection step for a possibly large scale MISO problem, thereby increasing the computational complexity of the identification algorithm to levels that are beyond feasibility. An alternative identification approach is presented employing regularized kernel-based methods. Keeping a parametric model for the module of interest, we model the impulse response of the remaining modules in the MISO structure as zero mean Gaussian processes (GP) with a covariance matrix (kernel) given by the first-order stable spline kernel, accounting for the noise model affecting the output of the target module and also for possible instability of systems in the MISO setup. Using an Empirical Bayes (EB) approach the target module parameters are estimated through an Expectation–Maximization (EM) algorithm with a substantially reduced computational complexity, while avoiding extensive model structure selection. Numerical simulations illustrate the potentials of the introduced method in comparison with the state-of-the-art techniques for local module identification.

Utilizing a unique high NA optical system, a new methodology to measure device overlay accurately has been developed with a key differentiation. Historically, optical techniques to measure features below the image resolution require supporting measurement techniques to be used as a reference to anchor the optical measurement. This novel selfreference methodology enables accurate and robust optical metrology for device features after etch eliminating the need for external reference measurements such as Decap, x-sections or high landing energy SEMs. In this paper, we discuss how a high NA Optical Metrology system enables measurements on small area device replica targets, which enables the ability to create a reference target for device measurements. The methodology utilizes this reference target to enable accurate direct on device overlay measurements without the need for an external reference. Furthermore, the technique is expanded to improve the robustness of the measurement and monitor live in production the health of the recipe, ensuring accuracy overtime. This ultimately leads to a method to extend the recipes in real-time based on the health KPIs. The improved accurate and robust device overlay measurements have proven to improve the overlay performance compared to other techniques. This, combined with the speed of optical systems, enables unconstrained dense measurements directly on device structures after etch, allowing for improved overlay control.

In this work a novel machine learning algorithm is used to calculate the after etch overlay of the memory holes in a 3D-NAND device based on OCD metrology by YieldStar S1375. It is shown that the method can distinguish the overlay signals from the process induced signals in the acquired pupil image and therefore, enables for an overlay metrology approach which is highly robust to process variations. This metrology data is used to characterize and correct the process induced intra-die stress and the DUV scanner application fingerprint.

In this article, we consider the identification of linear models from quantized output data. We develop a variational approximation of the likelihood function, which allows us to find variationally optimal approximations of the maximum-likelihood and maximum a posteriori estimates. We show that these estimates are obtained by projecting the midpoint in the quantization interval of each output measurement onto the column space of the input regression matrix. Interpreting the quantized output as a random variable, we derive its moments for generic noise distributions. For the case of Gaussian noise and Gaussian independent identically distributed input, we give an analytical characterization of the bias, which we use to build a bias-compensation scheme that leads to consistent estimates.

This work presents a systematic methodology to determine in real-time the optimal durations of the three stages of a new Ca-Cu looping process for H2 production with integrated CO2 capture. Economic and quality criteria are proposed to determine the appropriate time to switch between the stages. These criteria rely on the time-profiles of some key variables, such as product concentrations. Given the delayed nature of hardware sensor measurements, the real-time determination of such variables is based on soft-sensors. For this purpose, Gaussian Process Regression models are employed. The predictive capabilities of these models are tested on several datasets, yielding reliable predictions in most of the cases. The values of the optimal switching times computed with the proposed method differ from the actual values by 4 %, at most.

In networks of dynamic systems, one challenge is to identify the interconnection structure on the basis of measured signals. Inspired by a Bayesian approach in [1], in this paper, we explore a Bayesian model selection method for identifying the connectivity of networks of transfer functions, without the need to estimate the dynamics. The algorithm employs a Bayesian measure and a forward-backward search algorithm. To obtain the Bayesian measure, the impulse responses of network modules are modeled as Gaussian processes and the hyperparameters are estimated by marginal likelihood maximization using the expectation-maximization algorithm. Numerical results demonstrate the effectiveness of this method.

The identification of local modules in dynamic networks with known topology has recently been addressed by formulating conditions for arriving at consistent estimates of the module dynamics, typically under the assumption of having disturbances that are uncorrelated over the different nodes. The conditions typically reflect the selection of a set of node signals that are taken as predictor inputs in a MISO identification setup. In this paper an extension is made to arrive at an identification setup for the situation that process noises on the different node signals can be correlated with each other. In this situation the local module may need to be embedded in a MIMO identification setup for arriving at a consistent estimate with maximum likelihood properties. This requires the proper treatment of confounding variables. The result is an algorithm that, based on the given network topology and disturbance correlation structure, selects an appropriate set of node signals as predictor inputs and outputs in a MISO or MIMO identification setup. As a first step in the analysis, we consider the situation where the selected output node signals are predicted based on all of their in-neighbor node signals in the network.

We present a new class of models, called uncertain-input models, that allows us to treat system-identification problems in which a linear system is subject to a partially unknown input signal. To encode prior information about the input or the linear system, we use Gaussian-process models. We estimate the model from data using the empirical Bayes approach: the hyperparameters that characterize the Gaussian-process models are estimated from the marginal likelihood of the data. We propose an iterative algorithm to find the hyperparameters that relies on the EM method and results in decoupled update steps. Because in the uncertain-input setting neither the marginal likelihood nor the posterior distribution of the unknowns is tractable, we develop an approximation approach based on variational Bayes. As part of the contribution of the paper, we show that this model structure encompasses many classical problems in system identification such as Hammerstein models, blind system identification, and cascaded linear systems. This connection allows us to build a systematic procedure that applies effectively to all the aforementioned problems, as shown in the numerical simulations presented in the paper.

Boosting combines weak (biased) learners to obtain effective learning algorithms for classification and prediction. In this paper, we show a connection between boosting and kernel-based methods, highlighting both theoretical and practical applications. In the $$\ell _2$$ context, we show that boosting with a weak learner defined by a kernel K is equivalent to estimation with a special boosting kernel. The number of boosting iterations can then be modeled as a continuous hyperparameter, and fit (along with other parameters) using standard techniques. We then generalize the boosting kernel to a broad new class of boosting approaches for general weak learners, including those based on the $$\ell _1$$ , hinge and Vapnik losses. We develop fast hyperparameter tuning for this class, which has a wide range of applications including robust regression and classification. We illustrate several applications using synthetic and real data.

Identification of dynamic networks in prediction error setting often requires the solution of a non-convex optimization problem, which can be difficult to solve especially for large-scale systems. Focusing on ARMAX models of dynamic networks, we instead employ a method based on a sequence of least-squares steps. For single-input single-output models, we show that the method is equivalent to the recently developed Weighted Null Space Fitting, and, drawing from the analysis of that method, we conjecture that the proposed method is both consistent as well as asymptotically efficient under suitable assumptions. Simulations indicate that the sequential least squares estimates can be of high quality even for short data sets.

Generalized cross validation (GCV) is one of the most important approaches used to estimate parameters in the context of inverse problems and regularization techniques. A notable example is the determination of the smoothness parameter in splines. When the data are generated by a state space model, like in the spline case, efficient algorithms are available to evaluate the GCV score with complexity that scales linearly in the data set size. However, these methods are not amenable to on-line applications since they rely on forward and backward recursions. Hence, if the objective has been evaluated at time t−1 and new data arrive at time t, then O(t) operations are needed to update the GCV score. In this paper we instead show that the update cost isO(1), thus paving the way to the on-line use of GCV. This result is obtained by deriving the novel GCV filter which extends the classical Kalman filter equations to efficiently propagate the GCV score over time. We also illustrate applications of the new filter in the context of state estimation and on-line regularized linear system identification.

We address the problem of identifying a specific module in a dynamic network, assuming known topology. We express the dynamics by an acyclic network composed of two blocks where the first block accounts for the relation between the known reference signals and the input to the target module, while the second block contains the target module. Using an empirical Bayes approach, we model the first block as a Gaussian vector with covariance matrix (kernel) given by the recently introduced stable spline kernel. The parameters of the target module are estimated by solving a marginal likelihood problem with a novel iterative scheme based on the Expectation-Maximization algorithm. Numerical experiments illustrate the effectiveness of the proposed method.

In order to identify a specific system (module) of interest embedded in a dynamic network, one typically has to formulate a multi-input single-output (MISO) identification problem which requires to identify all modules in the MISO structure, and determine their model order. While the former task poses the problem of estimating a large number of parameters that are of no interest to the experimenter, the latter task may result computationally challenging in large-size networks. To avoid these issues and increase the accuracy of the identified module of interest, we use regularized kernel-based methods. Keeping a parametric model for the module of interest, we model the impulse response of the remaining modules in the MISO structure as zero mean Gaussian vectors with covariance matrix (kernel) given by the first-order stable spline kernel, accounting also for the noise model affecting the output of the target model. Using an Empirical Bayes (EB) approach, the target-module parameters are estimated by maximizing the marginal likelihood of the module output. The related optimization problem is solved using the Expectation-Maximization (EM) algorithm. Numerical experiments illustrate the potentials of the introduced method in comparison with the state-of-the-art techniques for local identification.

We propose a new methodology for identifying Wiener systems using the data acquired from two separate experiments. In the first experiment, we feed the system with a sinusoid at a prescribed frequency and use the steady state response of the system to estimate the static nonlinearity. In the second experiment, the estimated nonlinearity is used to identify a model of the linear block, feeding the system with a persistently exciting input. We discuss both parametric and nonparametric approaches to estimate the static nonlinearity. In the parametric case, we show that modeling the static nonlinearity as a polynomial results into a fast least-squares based estimation procedure. In the nonparametric case, least squares support vector machines (LS-SVM) are employed to obtain a flexible model. The effectiveness of the method is demonstrated through numerical experiments.

We analyze likelihood-based identification of systems that are linear in the parameters from quantized output data; in particular, we propose a method to find approximate maximum-likelihood and maximum-a-posteriori solutions. The method consists of appropriate least-squares projections of the middle point of the active quantization intervals. We show that this approximation maximizes a variational approximation of the likelihood and we provide an upper bound for the approximation error. In a simulation study, we compare the proposed method with the true maximum-likelihood estimate of a finite impulse response model.

We study the modeling of a stationary multivariate stochastic process as the output of a dynamic network driven by white noise. When this noise corresponds to the innovation, i.e. the unpredictable part of the process, we show that the network satisfies certain stability conditions. Restricting the network model to having diagonal noise structure, we show that the innovation-driven representation is unique and internally stable. We provide a one-to-one correspondence between this representation and the spectral factor associated with the innovation model. For two-node networks, we show that a representation with diagonal noise model can be obtained from a generic one through an explicit map.

We consider the problem of factorizing a matrix with discrete-valued entries as a product of two low-rank matrices. Under a probabilistic framework, we seek for the minimum mean-square error estimates of these matrices, using full Bayes and empirical Bayes approaches. In the first case, we devise an integration scheme based on the Gibbs sampler that accounts also for hyperparameter and noise variance estimation. A similar technique is used also for the latter case, where we combine Gibbs sampling with the expectation-maximization (EM) algorithm to estimate the model parameters via marginal likelihood maximization. Extension to the case of missing values is also discussed. The proposed methods are evaluated on simulated data, and on a real data set for recommender systems.

Adding input and output noises for increasing model identification error of finite impulse response systems is considered. This is motivated by the desire to protect the model of the system as a trade secret by rendering model identification techniques ineffective. Optimal filters for constructing additive noises that maximizes the identification error subject to maintaining the closed-loop performance degradation below a limit are constructed. Furthermore, differential privacy is used for designing output noises that preserve the privacy of the model.

We consider the problem of occupancy estimation in buildings using available environmental information. In particular, we study the problem of how to collect data that is informative enough for occupancy estimation purposes. We thus propose an application-oriented input design approach for designing the ventilation signal to be used while collecting the system identification datasets. The main goal of the method is to guarantee a certain accuracy in the estimated occupancy levels while minimizing the experimental time and effort. To take into account potential limitations on the actuation signals we moreover formulate the problem as a recursive nonlinear and nonconvex optimization problem, solved then using exhaustive search methods. We finally corroborate the theoretical findings with some numerical examples, which results show that computing ventilation signals using experiment design concepts leads to occupancy estimator performing 4 times better in terms of Mean Square Error (MSE).