In practical scenarios, collected data for identification may be contaminated by unexpected disturbances with large amplitudes. In such cases, the ordinary least squares estimator, commonly used for identification, may fail to deliver satisfactory performance. To address this issue, robust estimators that can withstand the influence of contaminated data become essential. This paper introduces the tilted least squares (TLS) robust estimator for handling outliers and heavy-tailed noises, which incorporates a weighted quadratic loss function, with the weights constrained by the Kullback-Leibler (KL) divergence. It is proposed that the TLS estimator assigns weights to each data point as an exponential function of the negative squared residuals, effectively mitigating the influence of unexpected disturbances with large amplitudes. Additionally, a tuning criterion is derived for automatically estimating the size of the KL divergence. Furthermore, it is demonstrated that a specific variant of the TLS estimator is equivalent to the relaxed least trimmed squares (RTLS) estimator and the almost sure convergence of the RTLS estimator is also established in the presence of heavy-tailed noises with infinite variance.

Delivering radiation therapy based on erroneous or corrupted treatment plan data has previously and unfortunately resulted in severe, sometimes grave patient harm. Aiming to prevent such harm and improve safety in radiation therapy treatment, this work introduces a novel, yet intuitive algorithm for strategically structuring the complex and unstructured data typical of modern treatment plans so their treatment sites may automatically be verified with deep-learning architectures. The proposed algorithm utilizes geometric and dose plan parameters to represent each plan’s data as a heat map to feed a deep-learning classifier that will predict the plan’s treatment site. Once it is returned by the classifier, a plan’s predicted site can be compared to its documented intended site, and a warning raised should the two differ.Using real head-neck, breast, and prostate treatment plan data retrieved at two hospitals in the United States, the algorithm is evaluated by observing the accuracy of convolutional neural networks (ConvNets) in correctly classifying the structured heat map data. Many well-known ConvNet architectures are tested, and ResNet-18 performs the best with a testing accuracy of 97.8% and 0.979 F-1 score. Clearly, the heat maps generated by the proposed algorithm, despite using only a few of the many available plan parameters, retain enough information for correct treatment site classification. The simple construction and ease of interpretation make the heat maps an attractive choice for classification and error detection.

Distributionally Robust Optimization (DRO), as a popular method to train robust models against distribution shift between training and test sets, has received tremendous attention in recent years. In this paper, we propose and analyze stochastic algorithms that apply to both non-convex and convex losses for solving Kullback Leibler divergence constrained DRO problem. Compared with existing methods solving this problem, our stochastic algorithms not only enjoy competitive if not better complexity independent of sample size but also just require a constant batch size at every iteration, which is more practical for broad applications. We establish a nearly optimal complexity bound for finding an $\epsilon$ stationary solution for non-convex losses and an optimal complexity for finding an $\epsilon$ optimal solution for convex losses. Empirical studies demonstrate the effectiveness of the proposed algorithms for solving non-convex and convex constrained DRO problems.

The least squares estimator is the most popular identification method. In the absence of prior knowledge on the unknown noise, uniform weights on all samples are often as-sumed. In reality, potentially unknown contamination is always present and the uniform weights are not necessarily the best. Further, explicit information about the nature of contamination is usually absent. To this end, a relaxed-tilted least squares method is proposed here to assign unequal weights so that the effect of undesired noise contamination can be mitigated. The relaxed-tilted least squares method tilts the uniform prior on the samples so as to move the uniform distribution in a direction that enjoys the smallest estimation error in the neighborhood of the uniform distribution. Theoretical results are established including the ability of outlier removal and the guaranteed parameter convergence in the presence of outliers. Numerical algorithms are proposed and simulated, which support the theoretical derivations.

In radiation oncology, the intricate process of delivering radiation to a patient is detailed by the patient's treatment plan, which is data describing the geometry, construction and strength of the radiation machine and the radiation beam it emits. The patient's life depends upon the accuracy of the treatment plan, which is left in the hands of the vendor-specific software automatically generating the plan after an initial patient consultation and planning with a medical professional. However, corrupted and erroneous treatment plan data have previously resulted in severe patient harm when errors go undetected and radiation proceeds. The aim of this paper is to develop an automatic error-checking system to prevent the accidental delivery of radiation treatment to an area of the human body (i.e., the treatment site) that differs from the plan's documented intended site. To this end, we develop a method for structuring treatment plan data in order to feed machine-learning (ML) classifiers and predict a plan's treatment site. In practice, a warning may be raised if the prediction disagrees with the documented intended site. The contribution of this paper is in the strategic structuring of the complex, intricate, and nonuniform data of modern treatment planning and from multiple vendors in order to easily train ML algorithms. A three-step process utilizing up- and down-sampling and dimension reduction, the method we develop in this paper reduces the thousands of parameters comprising a single treatment plan to a single two-dimensional heat map that is independent of the specific vendor or construction of the machine used for treatment. Our heat-map structure lends itself well to feed well-established ML algorithms, and we train-test random forest, softmax, k-nearest neighbors, shallow neural network, and support vector machine using real clinical treatment plans from several hospitals in the United States. The paper demonstrates that the proposed method characterizes treatment sites so well that ML classifiers may predict head-neck, breast, and prostate treatment sites with an accuracy of about 94%. The proposed method is the first step towards a thorough, fully automated error-checking system in radiation therapy.

This article considers identification of sparse Volterra systems. A method based on the almost orthogonal matching pursuit (AOMP) is proposed. The AOMP algorithm allows one to estimate one nonzero coefficient at a time until all nonzero coefficients are found without losing the optimality and the sparsity, thus avoiding the curse of dimensionality often encountered in Volterra system identification.

This paper investigates the uniqueness of parameters via persistence of excitation for switched linear systems. The main contribution is a much weaker sufficient condition on the regressors to be persistently exciting that guarantees the uniqueness of the parameter sets and also provides new insights in understanding the relation among different subsystems. It is found that for uniquely determining the parameters of switched linear systems, the needed minimum number of samples derived from our sufficient condition is much smaller than that reported in the literature.

In radiation therapy, preventing treatment plan errors is of paramount importance. In this paper, an alert system is proposed and developed for checking if the pending cancer treatment plan is consistent with the intended use. A key step in the development of the paper is characterization of various treatment plan fingerprints by three-dimension vectors taken from possibly thousands of variables in each treatment plan. Then three machine learning based algorithms are developed and tested in the paper. The first algorithm is a knowledge-based support vector machine method. If an incorrect treatment plan were offered, the algorithm would tell that the pending treatment plan is inconsistent with the intended use and provide a red flag. The algorithm is tested on the actual patient data sets with 100% successful rate and 0% failure rate. In addition, two algorithms based on the well-known k-nearest neighbour and Bayesian approach respectively are developed. Similar to the support vector machine algorithm, these two algorithms are also tested with 100% success rate and 0% failure rate. The key seems to pick up the right features.

Identification of a nonlinear nonparametric system is not easy. On the other hand, many systems are sparse in the sense that some variables do not contribute and some contribute only marginally. If these variables that do not contribute or contribute marginally can be detected and removed, the identification problem becomes lower dimensional and is relatively easy to deal with. The first goal of the paper is to develop an overlap group Lasso method to detect which variables contribute and which variables do not. The algorithm developed favors sparsity in terms of partial derivatives and provides a necessary and sufficient condition for a variable to contribute. Once contributing variables are identified, the second goal of the paper is to rank the importance of these variables based on the squared derivative averages. Since both the function and its derivatives are unknown, how to estimate the squared derivative averages is a concern. To this end, two methods are proposed with convergence results. The first one is nonparametric based on the Fourier transform of some intermediate variables. The other is to cast the problem in a Reproducing Kernel Hilbert Space (RKHS).

To achieve a parsimonious model, it is necessary to rank the importance of input variables according to some measures. The problem is nontrivial in the setting of nonlinear and nonparametric system identification. Difficulties lie in the lack of structural information of the unknown system, unknown underlying probabilistic distributions, and unknown nonlinear correlations of variables. In this article, we present a way to rank variables according to goodness of fit (GoF). Asymptotic results are established, and numerical algorithms are proposed. The problem is cast in a reproducing kernel Hilbert space (RKHS) that allows us to deal with nonparametric nature of the unknown system, to avoid making strong conditions on the unknown distributions, to link GoFs to computable conditional covariance operators on RKHS, and to develop computationally friendly numerical algorithms. Numerical simulations support the theoretical developments.

Focusing on identification, this paper develops techniques to reconstruct zero and nonzero elements of a sparse parameter vector θ of a stochastic dynamic system with general observation sequences, including stationary time series and feedback control, for which the current input may depend on the past inputs and outputs, system noises as well as exogenous dithers. First, a sparse parameter identification algorithm is introduced based on L2 norm with L1 regularization, where the adaptive weights are adopted in the optimization variables of L1 term. Second, estimates generated by the algorithm are shown to have both set and parameter convergence. That is, sets of the zero and nonzero elements in the parameter θ can be correctly identified with probability one using a finite number of observations, and further estimates of the nonzero elements converge to the true values almost surely. Third, it is shown that the results are applicable to a large number of applications, including variable selection, open-loop identification, and closed-loop control of stochastic systems. Finally, numerical examples are given to support the theoretical analysis.

Identification of a nonlinear nonparametric system is not easy.On the other hand, many systems are sparse in the sense that not all variables contribute.If these variables that do not contribute can be detected and removed, the identification problem becomes lower dimensional and is relatively easy to deal with.The goal of the paper is to develop an overlap group Lasso method to detect which variables contribute and which variables do not.The algorithm developed favors sparsity in terms of partial derivatives that is the necessary and sufficient condition for a variable to contribute.where () is the system output at time and () is an iid noise sequence of zero mean and finite variance, and is independent of the input variable (), = 1, . . ., .The regressor () = ( 1 (), . . ., ()) consists of possible contributing input variables.The structure of the nonlinear function is unknown.The system (1) represents a large class of nonlinear systems including the well known finite impulse response nonlinear systems [5] and the nonlinear autoregressive systems with exogenous inputs (NARX) [11,14], by letting () = (( -1), . . ., ( -)), () = (( -1), . . ., ( -), ( -1), . . ., ( -))CDSR 137-2 respectively.The goal of identification is to identify the unknown system (⋅) based on the available input-output data set {(), ()} =1 .As discussed, nonlinear system identification is not an easy task specially if the dimension is high.Fortunately for many practical applications, systems are sparse in the sense that not all variables contribute to the system.If the variables that do not contribute can be identified and removed, the dimension for identification could be smaller, also making the model parsimony. Overlap group Lasso approachIn this section, we propose a way to detect if the variable contributes or not for every .The objective is that the algorithm developed should not suffer from the curse of dimensionality like the local approach and should have a fixed number of unknown parameters to be estimated that does not increase as the data length increases as in both the local and RKHS approaches.To this end, assume that there are some basis functions ()'s, = 1,2, . . ., so that the unknown function (⋅) can be written asfor some > 0 and unknown coefficients * 's.Admittedly the assumption ( 2) is stronger than in RKHS representation () = ∑ ∞ =1 ((), ) that could have infinitely many terms.Thus, more prior information on the unknown (⋅) is needed.However, (2) does have applications for many practical systems.For instance suppose is unknown but is known that it is a polynomial of the maximum .Then the assumption ( 2) is valid and the choice of () is obvious.Now, decompose the coefficients set { 1 * , . . ., * } into subsets 1 , . . ., , = { * | () , = 1,2, . . ., }, = 1,2, . . ., Clearly ⋃ =1 ⊂ { 1 * , . . ., * }, but there could be overlaps between and .If no confusion we may also refer by the indices of * ∈ , i.e., = {| () }.We now make a simple but important observation: ≡ 0 ⟺ * = 0 * ∈ ⟺ ∑ ∈ * 2 = 0 We give an example here.Consider a polynomial of 1 , 2 , 3 of order 2, () = 1 * 1 + 2 * 2 + 3 * 3 + 4 * 1 2 + 5 * 2 2 (3) + 6 * 3 2 + 7 * 1 2 + 8 * 2 3 + 9 * 2 3 = 1 * 1 () + 2 * 2 () + 3 * 3 () + 4 * 4 () + 5 * 5 () + 6 * 6 () + 7 * 7 () + 8 * 8 () + 9 * 9 ()Then,

In this paper, we consider the variable selection for linear stochastic systems. A modified LASSO-type estimator is introduced. Then based on the classical persistent excitation (PE) for systems identification, the strong consistency of the estimates is established. i.e.. the zero elements in the unknown parameter vector being correctly identified and estimates for the nonzero elements in the unknown parameter vector converging to the true values with probability one. Compared with the existing results on similar topics, the strong consistency of estimates is established while in existing literature only the convergence in probability was obtained. For this, new theoretical analysis method is adopted in this paper.

Focusing on identification, this paper develops techniques to reconstruct zero and nonzero elements of a sparse parameter vector of a stochastic dynamic system under feedback control, for which the current input may depend on the past inputs and outputs, system noises as well as exogenous dithers. First, a sparse parameter identification algorithm is introduced based on L2 norm with L1 regularization, where the adaptive weights are adopted in the optimization variables of L1 term. Second, estimates generated by the algorithm are shown to have both set and parameter convergence. That is, sets of the zero and nonzero elements in the parameter can be correctly identified with probability one using a finite number of observations, and estimates of the nonzero elements converge to the true values almost surely. Third, it is shown that the results are applicable to a large number of applications, including variable selection, open-loop identification, and closed-loop control of stochastic systems. Finally, numerical examples are given to support the theoretical analysis.

Depression is a serious problem for many older adults but is too often undetected by the person, family or providers. Although vocal patterns have been successfully used to detect and predict depression in adults aged 18 to 65 years, no studies to date have included older adults. The study purpose was to determine whether vocal patterns associated with clinical depression in younger people also signify depression in older adults. An observational, repeated measures design was used to enroll 46 volunteer older adults who completed a semi-structured interview composed the 9-item Patient Health Questionnaire or PHQ-9 depression scale and selected speech measures. Recorded interviews were analysed by machine learning algorithms to evaluate whether vocal patterns may predict presence of depression in older adults. In this study, using the PHQ-9 and a supervised machine learning algorithm accurately predicted high and low depression scores between 86% and 92% of the time. Change in raw PHQ-9 scores between interview cycles was predicted within 1.17 points. These results provide strong and promising evidence that vocal patterns can be used effectively to detect clinical depression in adults who are 65 years and older.

Probabilistic wind power forecasting has become an important tool for optimal economic dispatch and unit commitment of modern power systems with significant renewable energy penetrations. Ensemble forecasting based on Monte Carlo simulation has been widely adopted by grid operators, but other probabilistic approaches, such as multistep iterative wind power forecasting have not yet been fully explored. The associated uncertainty analysis is an important yet challenging issue in this area. This paper proposes to use an analytic interval forecasting framework to estimate the forecasting uncertainty and its propagation with multisteps for two wind farms based on the temporally local Gaussian process (TLGP) model. The key findings confirm that TLGP forecasting not only has better accuracy but is also more reliable and sharp than other benchmark models. This paper provides an innovative analytical framework for iterative multistep interval forecasts.

A parsimonious model is always preferred in engineering applications not only because it has a better prediction ability but also because it suffers less from the curse of dimensionality in data-based modeling. One way to achieve a parsimonious model is to identify contributing variables from the candidate variables and then to eliminate noncontributing or redundant variables. However, identifying which variables contribute and which variables do not contribute is not an easy task for a nonparametric nonlinear system. This paper considers variable-selection problems for a nonlinear nonparametric system. Two approaches, inverse and contour variable-selection algorithms, are proposed along with their theoretical analysis and numerical algorithms. Neither approach suffers from the curse of dimensionality, which is usually a problem for traditional variable-selection methods for a nonparametric nonlinear system. Furthermore, no elliptic symmetry nor independent input variables are assumed, so both algorithms enjoy wide applications. Numerical algorithms for both approaches are fairly straightforward and simple.

The importance of discovering significant variables from a large candidate pool is now widely recognized in many fields. There exist a number of algorithms for variable selections in the literature. Some are computationally efficient but only provide a necessary condition, not a sufficient and necessary condition, for testing if a variable contributes or not. The others are computationally expense. The goal of the paper is to develop a directional variable selection algorithm that performs similar to or better than the leading algorithms for variable selections, but under weaker technical assumptions and with a much reduced computational complexity.

Additive nonlinear systems are one of the most widely used nonlinear and non-parametric models to describe nonlinear behaviors. A number of analysis and identification techniques have been developed in the literature for such systems. To apply, however, one has to make sure that the system is additive or can be approximated well by an additive system. This is a nontrivial problem and has eluded researchers for a long time. To the best of our knowledge, only scatter results are reported in the literature. The difficulties lie in the fact that the function and its structure are unknown so estimating its derivatives under an unknown multidimensional density function could be subject to the curse of dimensionality. In this paper, we present two methods to check if a system is additive. The first one estimates the squared derivative average via the Fourier transform. The other one directly estimates the squared derivative average in a reproducing kernel Hilbert space (RKHS) setting. For both methods, convergence results are established and practical numerical algorithms are developed.