The Gaussian kernel is one of the most important kernels, applicable to many research fields, including scientific computing and data science. In this paper, we present asymptotic analysis of the Gaussian kernel matrix in high dimension under a statistical model of noisy data. The main result is a nice combination of Karoui's asymptotic analysis with procedures of constrained low rank matrix approximations. More specifically, Karouli clarified an important asymptotic structure of the Gaussian kernel matrix, leading to strong consistency of the eigenvectors, though the eigenvalues are inconsistent. This paper focuses on the above results and presents a consistent estimator with the use of the smallest eigenvalue, whenever the target kernel matrix tends to low rank in the asymptotic regime. Importantly, asymptotic analysis is given under a statistical model representing partial noise. Although a naive estimator is inconsistent, applying an optimization method for low rank approximations with constraints, we overcome the difficulty caused by the inconsistency, resulting in a new estimator with strong consistency in rank deficient cases.

Matrix computations have a long history of study in view of numerous applications in scientific and engineering research fields. Nowadays, it is important to develop numerical algorithms for observation matrices contaminated with random noise arising in research fields such as data science and machine learning. This study presents a unified view of considerable statistical applications relevant to the singular value decomposition with orthogonal projections for the observation matrices. In particular, we focus on the errors-in-variables linear regression model and the dynamic mode decomposition with observation errors to clarify important matrix structures of statistical estimators. From this perspective, we provide a general framework for consistency analysis of statistical estimators computed from the singular value decomposition using orthogonal projections. Our asymptotic matrix analysis leads to a basic principle for the construction of consistent estimators using the orthogonal projections, resulting in an extension of the existing estimators. Moreover, strong consistency of the estimators can be proved straightforwardly under reasonable conditions. It is worth noting that our framework of consistency analysis covers a complicated structured problem related to the Vandermonde matrices that arise in important real-world applications.

In this paper, we prove strong consistency of an estimator by the truncated singular value decomposition for a multivariate errors-in-variables linear regression model with collinearity. This result is an extension of Gleser's proof of the strong consistency of total least squares solutions to the case with modern rank constraints. While the usual discussion of consistency in the absence of solution uniqueness deals with the minimal norm solution, the contribution of this study is to develop a theory that shows the strong consistency of a set of solutions. The proof is based on properties of orthogonal projections, specifically properties of the Rayleigh-Ritz procedure for computing eigenvalues. This makes it suitable for targeting problems where some row vectors of the matrices do not contain noise. Therefore, this paper gives a proof for the regression model with the above condition on the row vectors, resulting in a natural generalization of the strong consistency for the standard TLS estimator.

In this paper, we construct an estimator of an errors-in-variables linear regression model. The regression model leads to a constrained total least squares problems with row and column constraints. Although this problem can be numerically solved, it is unknown whether the solution has consistency in the statistical sense. The proposed estimator can be constructed by the use of orthogonal projections and their properties, its strong consistency is naturally proved. Moreover, our asymptotic analysis proves the strong consistency of the total least squares solution of the problem with row and column constraints.

Dynamic mode decomposition (DMD) is known as an efficient analysis method for time series data. We introduce an adaptive averaging technique to the DMD for its strong convergence in the statistical sense under a newly constructed noise model. The new model can represent time series data contaminated with random noise in more general situation than the existing one. However, due to its generality, the model itself is complicated, which makes statistical asymptotic analysis less straightforward. As it stands, under the new statistical model, there is no convergence proof of any existing method as far as the author knows. Although the proposed averaging technique is simple and not so new, it is worth noting that this technique can be generally applied to the above new noise model as the preconditioning, resulting in natural proof of the strong convergence. Moreover, we prove that an estimator of the original DMD is not consistent in the statistical sense, which implies the importance of the averaging technique as the preconditioning.

In this paper, we construct a natural errors-in-variables regression model corresponding to the total least squares (TLS) problem with linear equations constraints and prove strong consistency of the estimation. The presented asymptotic theory in the statistical sense is a generalization of the consistency analysis of an estimation with the use of ordinary total least squares for basic errors-in-variables regression models.

Dynamic mode decomposition (DMD) has attracted much attention in recent years as an analysis method for time series data. In this paper, we perform asymptotic analysis on the DMD to prove strong consistency in the statistical sense. More specifically, we first give a statistical model of random noise for data with observation noise. Among many variants of the DMD, the total least squares DMD (TLS-DMD) is known as a robust method for the random noise in the observation. We focus on consistency analysis of the TLS-DMD in the statistical sense and aim to generalize the analysis for projected methods. This paper gives a general framework for designing projection methods based on efficient dimensionality reduction analogously to the proper orthogonal decomposition under the statistical model and proves its strong consistency of the estimation.

We propose a refinement algorithm for singular value decomposition (SVD) of a real matrix. In the same manner as Newton’s method, the proposed algorithm converges quadratically if a modestly accurate initial guess is given. Since the proposed algorithm is based on matrix multiplication, it can efficiently be implemented. Numerical results demonstrate the excellent performance of the proposed algorithm in terms of the convergence rate and the measured computing time compared to a standard approach using multiple precision arithmetic.

Dynamic mode decomposition (DMD) is a popular technique for extracting important information of nonlinear dynamical systems. In this paper, we focus on the DMD based on the total least squares (TLS), which is experimentally efficient for noisy datasets for a dynamical system, while the asymptotic analysis is not given. We propose a statistical model of random noise, adapting to the Koopman operator associated with the DMD. Moreover, under reasonable assumptions, we prove strong convergence of random variables, corresponding to the eigenpairs computed by the DMD based on the TLS.

We consider numerical methods for computing eigenvalues located in the interior part of the spectrum of a large symmetric matrix. For such difficult eigenvalue problems, an effective solution is to use the Harmonic Ritz pairs in projection methods because the error bounds on the Harmonic Ritz pairs are well studied. In this paper, we prove global convergence of the iterative projection methods with the Harmonic Ritz pairs in an abstract form, where the standard restart strategy is employed. To this end, we reformulate the existing convergence proof of the Ritz pairs to be successfully applied to the Harmonic Ritz pairs with the inexact linear system solvers. Our main theorem obtained by the above convergence analysis shows important features concerning the global convergence of the Harmonic Ritz pairs.

Inverse eigenvalue problems arise in a variety of applications, and thus various Newton's methods, which quadratically converge, have been developed both in theory and practice. Among many studies over thirty years, two extremely significant developments are found. Firstly, smooth matrix decompositions have been successfully applied since the 1990s. Secondly, a matrix multiplication based method has been recently proposed. In this paper, such efficient modern solvers are classified in the context of classical Newton's methods according to their mathematical formulations, and then the corresponding convergence theorems and their relationship are surveyed.

In 2018, for inverse generalized symmetric eigenvalue problems, a new quadratically convergent algorithm was discovered from simple matrix equations. Although this algorithm has some nice features compared with the standard Newton’s method, it cannot be applied to multiple eigenvalues. In this paper, we propose a modified algorithm adapted to an arbitrary set of prescribed eigenvalues. Moreover, we prove its quadratic convergence in a neighborhood of the solutions that satisfy a mild condition.

An efficient refinement algorithm is proposed for symmetric eigenvalue problems. The structure of the algorithm is straightforward, primarily comprising matrix multiplications. We show that the proposed algorithm converges quadratically if a modestly accurate initial guess is given, including the case of multiple eigenvalues. Our convergence analysis can be extended to Hermitian matrices. Numerical results demonstrate excellent performance of the proposed algorithm in terms of convergence rate and overall computational cost, and show that the proposed algorithm is considerably faster than a standard approach using multiple-precision arithmetic.

We propose a quadratically convergent algorithm for inverse symmetric eigenvalue problems based on matrix equations. The basic idea is seen in a recent study by Ogita and Aishima, while they derive an efficient iterative refinement algorithm for symmetric eigenvalue problems using special matrix equations. In other words, this study is interpreted as a unified view on quadratically convergent algorithms for eigenvalue problems and inverse eigenvalue problems based on matrix equations. To the best of our knowledge, such a unified development of algorithms is provided for the first time. Since the proposed algorithm for the inverse eigenvalue problems can be regarded as the Newton's method for the matrix equations, the quadratic convergence is naturally proved. Our algorithm is interpreted as an improved version of the Cayley transform method for the inverse eigenvalue problems. Although the Cayley transform method is one of the effective iterative methods, the Cayley transform takes O(n3) arithmetic operations to produce an orthogonal matrix using a skew-symmetric matrix in each iteration. Our algorithm can refine orthogonality without the Cayley transform, which reduces the operations in each iteration. It is worth noting that our approach overcomes the limitation of the Cayley transform method to the inverse standard eigenvalue problems, resulting in an extension to inverse generalized eigenvalue problems.

In 2017, for inverse symmetric eigenvalue problems, a new quadratically convergent algorithm has been derived from simple matrix equations. Although this algorithm has some nice features compared with the other quadratically convergent methods, it is not applied to multiple eigenvalues. In this paper, we improve this algorithm with the aid of an optimization problem for the eigenvectors associated with multiple eigenvalues. The proposed algorithm is adapted to an arbitrary set of given eigenvalues. The main contribution is our convergence theorem formulated in a different manner from previous work for the existing quadratically convergent methods. Our theorem ensures the quadratic convergence in a neighborhood of the solutions that satisfy a mild condition.

In 2015, Gower and Richtarik presented a unifying framework for a variety of randomized iterative algorithms for consistent linear systems. The framework includes the randomized Kaczmarz method that exponentially converges in the mean square whenever the system is consistent. For noisy linear systems corresponding to inconsistent systems, the randomized Kaczmarz method computes an approximate solution within a fixed distance depending on the norm of the noise vector. We extend this error analysis to a general framework in inconsistent systems in a similar manner to Gower and Richtarik, and verify this theoretical analysis in numerical experiments.

We prove global convergence of particular iterative projection methods using the so-called shift-and-invert technique for solving symmetric generalized eigenvalue problems. In particular, we aim to provide a variant of the convergence theorem obtained by Crouzeix, Philippe, and Sadkane for the generalized Davidson method. Our result covers the Jacobi–Davidson and the rational Krylov methods with restarting and preconditioning that are important techniques for modern eigensolvers. More specifically, we prove that the Ritz pairs converge to exact eigenpairs, even though they are not necessarily the target eigenpairs. We would like to emphasize that our proof is not a routine consideration of Crouzeix, Philippe, and Sadkane. To complete the proof, we discover a key lemma, which leads to a very simple convergence proof, resulting in a new theorem similar to that of Crouzeix, Philippe, and Sadkane.

In this paper, We present the generalizations of the LSMR and NSCG methods for solving matrix equations. First, based on the LSMR algorithm, the Bl-LSMR and Gl-LSMR algorithms are derived by minimizing the Frobenius norm of residual matrix of normal equation. In addition, by extending the idea of LSMR algorithm, we also present the LSMR-M algorithm for solving the general coupled matrix equations. Next, based on NSCG and NS-CGNR methods, we establish the iterative methods which are inner/outer iterations for solving the sylvester equation and matrix equation AXB = C. Convergence conditions of each method are studied in dept and by using the numerical experiments the efficiency of the methods versus some well-known iterative method are shown. We also show that the Hermitian splitting and quasi-Hermitian splitting can induce accurate, robust, and effective preconditioned Krylov subspace methods. The Extended Abstracts of The 8 Seminar on Linear Algebra and its Applications 13-14th May 2015, University of Kurdistan, Iran STRUCTURED PERTURBATIONS

The Lanczos method is well known for computing the extremal eigenvalues of symmetric matrices. For efficiency and robustness a restart strategy is employed in practice, but this makes an analysis of convergence less straightforward. We prove global convergence of the restarted Lanczos method in exact arithmetic using certain convergence properties of the Rayleigh–Ritz procedure, which can be obtained from the discussion by Crouzeix, Philippe, and Sadkane. For the restarted Lanczos method, Sorensen’s previous analysis establishes global convergence to the largest eigenvalues under the technical assumption that the absolute values of the off-diagonal elements of the Lanczos tridiagonal matrix are larger than a positive constant throughout the iterations. In this paper, we prove global convergence without any such assumption. The only assumption is that the initial vector is not orthogonal to any of the target exact eigenvectors. More importantly, our results are applicable to dynamic restarting procedures where the dimensions of the projection subspace are dynamically determined. In other words, our analysis can be applied to the recently proposed efficient restart strategies employed in the thick restarted Lanczos method. The convergence theorem is extended to the restarted Lanczos method for computing both the largest and smallest eigenvalues. Moreover, we derive certain global convergence theorems of the block Lanczos and Jacobi–Davidson methods, where, for both algorithms, the Ritz values are shown to converge to exact eigenvalues, although they are not necessarily extremal.

We consider convergence and a posteriori error estimates of the classical Jacobi method for solving symmetric eigenvalue problems. The famous convergence proof of the classical Jacobi method consists of two phases. First, it is shown that all the off-diagonal elements converge to zero. Then, from a perturbation theorem, Parlett or Wilkinson shows convergence of the diagonal elements in the textbooks. Ciarlet also gives another convergence proof based on a discussion about a bounded sequence corresponding to a diagonal element. In this paper, we simplify the Ciarlet’s convergence proof. Our proof does not use any perturbation theory. Moreover, employing this approach, we obtain a posteriori error estimates for eigenvectors.