In linear models having near collinear columns of X, ridge and surrogate estimators often are used to mitigate collinearity. A new class of estimators is based on mixtures, either of X and a design minimal in an ordered class or of the Fisher information and a scalar matrix. Comparisons are drawn among choices for the mixing parameter, and the estimators are found to be admissible relative to ordinary least squares. Case studies demonstrate that selected mixture designs are perturbed from the original design to a lesser extent than are those of the surrogate method, while retaining reasonable efficiency characteristics.

Given observations { Y i ; 1 ≤ i ≤ n { with dispersion matrix Σ , a pervading issue is whether shifts have occurred in designated subsets of the observations. Early work on single shifts used order statistics or the R-Student t i statistics as diagnostics, initially derived under i.i.d. Gaussian assumptions. These diagnostics recently have been shown to remain exact in level and power under equicorrelation and more general dispersion structures, and under star-contoured mixtures supplanting Gaussian errors, with an accounting for irregularities engendered by shifts at other than the designated cases. Extensions here pertain to outlying subsets using the R -Fisher diagnostics F I , showing invariance of its distribution and of related diagnostics under more general dispersion structures and mixtures over these. Shifts occurring at cases other than those designated induce doubly noncentral F distributions. These elicit profound disturbances in operating characteristics of the diagnostics, serving in turn to explain masking and swamping, and the discovery of hidden “regression effects” among outliers. Evidence for anomalies arising from denominator noncentralities rests on two-sided rejection rules to be given. Numerical studies serve to illuminate the essence of the findings in practice.

Given a model {Y = X beta + epsilon} with Fisher information matrix Xi = X'X, a principal objective is to find information enhancing transformations T for which T Xi T' >=(L) Xi under the positive definite ordering, so as to improve essentials in linear inference. This is achieved through properties of congruences together with basic orderings of linear spaces. These foundations in turn support a new class of geometric mixture models on "mixing" the original design with another to assume the role of "target," to the following effects. Ridge, surrogate, and other solutions are often used to mitigate the effects of ill-conditioned models. Instead, in this study an ill-conditioned design matrix X is mixed with a well-conditioned design as target, leveraging the former toward the latter as the mixing parameter evolves, thus offering an alternative approach to ill-conditioning. The methodology is demonstrated with case studies from the literature, where the geometric mixtures are compared with the ridge, surrogate, and recently found arithmetic mixture models.

Lower and upper spectral bounds are known for matrices X X(k x k) under Loewner [Uber monotone matrixfunktionen. Math Z. 1934; 38: 177-216] order, as are corresponding bounds for the factor X(n x k) under an induced order. Least upper bounds for the latter give designs with dominating Fisher Information, with consequent gains in linear inference; see Jensen DR, Ramirez DE [Enhanced design efficiency through least upper bounds. J Stat Comput Simul. 2016; 86: 1798-1817]. The present study examines properties on ordering the singular values of a design matrix using majorization as in Marshall and Olkin [Inequalities: theory of majorization and its applications. New York: Academic Press; 1979]. The principal focus includes conditioning through condition numbers, variance inflation factors, and lengths and efficiencies of OLS solutions. Functions monotone under the induced order are identified; equivalence classes of designs are displayed preserving a dispersion matrix or its eigenvalues; a minimal element Xm(n x k) is characterized; as are equivalence classes of (A, D, E)optimal designs showing the latter not to be unique. Algorithms to achieve enhanced designs are given on modifying a single design, or on amalgamating two designs, with essential consequences in linear inference. A collateral procedure, based on mixtures of Fisher information matrices, serves effectively to ameliorate the ill effects of near collinearity. Case studies illustrate gains to be made in practice, to include a substantial improvement in the analysis of classically ill-conditioned data from the literature.

Given Gaussian observation vectors $[\seqcl{\BY}{n}]$ having a common mean and dispersion matrix, a pervading issue is to identify shifted observations of type $\{\BYi\!\to\!\BYi\!+\!\bdeli\}.$ Conventional usage enjoins Hotelling's $\Tisq$ diagnostics, derived and applied under the mutual independence of $[\seqcl{\BY}{n}]$. Independence often fails, yet the need to identify outliers nonetheless persists. Accordingly, the present study reexamines $\Tisq$ under dependencies to include equicorrelations and more general matrices. Such dependencies are found in the analysis of calibrated vector measurements and elsewhere. In addition, mixtures of these distributions having star--shaped contours arise on occasion in practice. Nonetheless, the $\Tisq$ diagnostics are shown to remain exact in level and power for all such mixtures. Moreover, further matrix distributions, not necessarily having finite moments, are seen to generalize $n$--dimensional spherical symmetry to include non--Gaussian matrices of order $(n\!\times\!k)$ supporting $\Tisq.$ For these the use of $\Tisq$ remains exact in level. These findings serve to expand considerably the range of applicability of $\Tisq$ in practice, to include matrix Cauchy and other heavy tailed distributions intrinsic to econometric and other studies. Case studies serve to illuminate the methodology.

Collinearity in the design matrix is a frequent problem in linear regression models, for example, with economic or medical data. Previous standard procedures to mitigate the effects of collinearity include ridge regression and surrogate regression. Ridge regression perturbs the moment matrix , while surrogate regression perturbs the design matrix . More recently, the raise estimators have been introduced, which allow the user to track geometrically the perturbation in the data with . The raise estimators are used to reduce collinearity in linear regression models by raising a column in the experimental data matrix, which may be nearly linear with the other columns, while keeping the basic OLS regression model. We give a brief overview of these three ridge-type estimators and discuss practical ways of choosing the required perturbation parameters for each procedure.

Lower and upper spectral bounds are known for positive-definite (k x k) matrices in (S-k(+), >=(L)) under Loewner (Uber monotone Matrixfunktionen. Math Z. 1934; 38: 177-216) ordering. Lower and upper singular bounds for matrices of order (n x k) in (F-nxk, >=) derive under an induced ordering. These orderings are combined here to the following effects. Given two first-order experimental designs (X, Z) in (F-nxk, >=), their upper singular bound XM enhances both X and Z in that its Fisher Information matrix dominates those for both X and Z, thus ordering essentials in Gauss-Markov estimation. Moreover, if Sigma, Omega, and Xi are dispersion matrices for linear estimators under X, Z, and X-M, respectively, then Xi is the spectral lower bound for (Sigma, Omega) in (S-k(+), >=(L)). In essence this algorithm identifies elements in Z complementary to those of X, and combines these into X-M. Case studies illustrate gains to be made thereby in first and second-order designs. Specifically, two examples demonstrate that designs optimal under separate criteria may be combined into a single design dominating both. In addition, selected examples demonstrate that classical second-order designs may be improved inter se.

The raised estimators are used to reduce collinearity in linear regression models by raising a column in the experimental data matrix which may be nearly linear with the other columns. The raising procedure has two components, namely stretching and rotating, which we can analyze separately. We give the relationship between the raised estimators and the classical ridge estimators. Using a case study, we show how to determine the perturbation parameter for the raised estimators by controlling the amount of precision to be retained in the original data.

Consider [Y1,…, Yn] as Gaussian observations with common mean μ and dispersion matrix Σ. Approaches for detecting outlying observations include the R-Student statistics in regression diagnostics, as well as tests due to Grubbs, Dixon, and Ferguson using order statistics. All are known to be valid under Σ = σ2In; Grubbs’s test also holds under an equicorrelated matrix Σ(ρ) and the more general structure \(\sum \,(\xi ) = {\sigma ^2}[{I_n} + {1_n}\xi ' + \xi {1'_n} - \bar \xi {1_n}{1'_n}]\). Dispersion mixtures of Gaussian errors having Σ(ρ) and Σ(ξ) are studied in detail; their densities have star-shaped contours as encountered on occasion in practice. Under these mixtures, the aforementioned diagnostics all are shown to be exact in significance level and in power as for the case where Σ = σ2In. This expands considerably their range of applicability in practice. Case studies serve to illustrate essentials of the findings.

Corrections are given to common misconceptions regarding the use of centered and uncentered regressors, and of meanings to be ascribed to A -, D -and E -criteria in evaluating two such designs.

We study response surface designs using the generalized variance inflation factors for subsets as an extension of the variance inflation factors.

Shifts in responses typically are obscured from users, so that regression proceeds as if unshifted. At issue is the infusion of such shifts into classical analysis. On projecting outliers into the “Regressor” and “Error” spaces of a model, findings here are that shifts in responses may account for shifts in the OLS solutions, or for inflated residuals, or both. These in turn impact estimation, prediction, and hypothesis tests, all of vital interest to users, and all considered here. Tools for identifying shifts are given. Case studies illustrate effects of shifts on regression, to include a reexamination of studies from the literature.

Anomalies persist in the use of deletion diagnostics in regression. Tests for outliers under subset deletions utilize the R-Fisher FI statistics, each having a noncentral F-distribution with noncentrality parameter λ as a function of shifts only at deleted rows in the index set I. Numerous studies examine empirical outcomes of these diagnostics in random experiments. In contrast, studies here are probabilistic, examining distributions behind those empirical outcomes and tracking the effects of shifts at nondeleted rows. By allowing shifts at nondeleted rows in a set J, in addition to traditional shifts at deleted rows in I, FI is shown to have a doubly noncentral F-distribution. By removing the unnecessary restriction that shifts occur only at deleted rows, these findings support constructs akin to power curves in tracking probabilities of masking or swamping as shifts evolve. In addition, “regression effects” among outliers may have unforeseen consequences. A dichotomy of shifts is discovered as projections into the “regressor” and “error” spaces of a model. Hidden shifts at nondeleted rows can obfuscate not only meanings ascribed to traditional outlier diagnostics, but also to subset influence diagnostics corresponding one-to-one with FI. In short, despite wide usage abetted by software support, deletion diagnostics in current vogue no longer can be recommended to achieve objectives traditionally cited. Case studies illustrate the debilitating effects of these anomalies in practice, together with conclusions misleading to prospective users.

Variance Inflation Factors (VIFs) are reexamined as conditioning diagnostics for models with intercept, with and without centering regressors to their means as oft debated. Conventional VIFs, both centered and uncentered, are flawed. To rectify matters, two types of orthogonality are noted: vector-space orthogonality and uncorrelated centered regressors. The key to our approach lies in feasible Reference models encoding orthogonalities of these types. For models with intercept it is found that (i) uncentered VIFs are not ratios of variances as claimed, owing to infeasible Reference models; (ii) instead they supply informative angles between subspaces of regressors; (iii) centered VIFs are incomplete if not misleading, masking collinearity of regressors with the intercept; and (iv) variance deflation may occur, where ill-conditioned data yield smaller variances than their orthogonal surrogates. Conventional VIFs have all regressors linked, or none, often untenable in practice. Beyond these, our models enable the unlinking of regressors that can be unlinked, while preserving dependence among those intrinsically linked. Moreover, known collinearity indices are extended to encompass angles between subspaces of regressors. To reaccess ill-conditioned data, we consider case studies ranging from elementary examples to data from the literature.

Let X be an input measurement and Y the output reading of a calibrated instrument, with Y(X) as the calibration curve. Solving X(Y) projects an instrumental reading back onto the scale of measurements as an object of pivotal interest. The arrays of instrumental readings are projected in this manner in practice, yielding arrays of calibrated measurements, typically subject to errors of calibration. The effects of calibration errors on the properties of calibrated measurements are examined here under linear calibration. Irregularities arise as induced dependencies, inflated variances, non-standard distributions, inconsistent sample means, the underestimation of measurement variance, and other unintended consequences. On the other hand, conventional properties are seen to remain largely in place in the use of selected regression diagnostics and in one-way comparative experiments using calibrated data.

Ridge regression, perturbing the design moment matrix via a parameter k, persists in the study of ill-conditioned systems. Ridge traces, exhibiting solutions as functions of k, are intended to reflect stability as k evolves, in contrast to transient instabilities in ordinary least squares. This study examines derivative traces as analytic tools regarding stability, and develops rational representations for them. Two further gauges of stability are derivatives of variances of the ridge solutions, and the variances of the derivative traces. In contrast to ridge traces and their derivatives, neither of the latter depends on observed responses, and both support deterministic assessments.

In O’Driscoll and Ramirez (2011a), it was noted (as expected) that the bias of the MLE estimator of the slope in the measurement error regression model is monotone decreasing as the estimated variances error ratio e approaches the true variances error ratio � = � 2 �/� 2 �. However for a fixed estimated variances

The slope of the best fit line from minimizing the sum of the squared oblique errors is the root of a polynomial of degree four. This geometric view of measurement errors is used to give insight into the performance of various slope estimators for the measurement error model including an adjusted fourth moment estimator introduced by Gillard and Iles (2005) to remove the jump discontinuity in the estimator of Copas (1972). The polynomial of degree four is associated with a minimun deviation estimator. A simulation study compares these estimators showing improvement in bias and mean squared error.

Ridge versions of an ill-conditioned system are alleged to “act more like an orthogonal system” than the system itself. Alternatives, called surrogates and based on the conditioning of linear systems, are shown to yield smaller expected mean squares than OLS, and uniformly smaller residual sums of squares than ridge. Ridge and surrogate solutions are compared on several marques of orthogonality to include conditioning of dispersion arrays, variance inflation factors, isotropy of variances, and sphericity of contours of the estimators. For these, ridge typically exhibits erratic divergence from orthogonality as the ridge scalar evolves, often reverting back to OLS in the limit. In contrast, surrogate solutions converge monotonically to those from orthogonal systems. Invariance considerations constrain the computations to models in canonical form. Case studies serve to illustrate the central issues.