This paper addresses the solution of the inverse eigenvalue problems for tensors represented in tensor-train (TT) format with prescribed TT-ranks. Leveraging the inherent structural properties of the TT format and the tensor-vector product, we reformulate the problem mentioned above as a multi-objective optimization problem, and an alternating iterative method, derived from the classical alternating least-squares (ALS) approach, is presented. Particularly, in each subproblem a proximal regularization term is specifically designed to enhance the numerical robustness of the proposed method. It is proven that all limit points of the iterative sequences generated by our method satisfy the KKT conditions for the underlying optimization problem. Furthermore, under certain assumptions, we prove that this method achieves Q-linear convergence. Numerical examples are provided to illustrate the feasibility and efficiency of the proposed method.
Stochastic multi-objective bilevel optimization (SMOBO) has emerged as a powerful framework for large-scale machine learning problems, where stochastic gradient methods are naturally employed for their simplicity and efficiency. However, existing stochastic gradient methods for SMOBO typically require computing the expensive inverse Hessian matrices, leading to high computational cost. To tackle these issues, we propose a fully Hessian-inverse-free stochastic gradient method for SMOBO, which is computationally efficient due to the use of a decoupling approach. Specifically, we employ a momentum-like gradient estimate to replace the stochastic gradients with their moving average, thereby mitigating the errors induced by dynamic weight updates. Without assuming bounded function values, we demonstrate that the proposed method achieves not only an optimal non-asymptotic convergence rate of 𝒪(K^-1/2) for SMOBO, but also an improved bias bound for the stochastic multi-gradient with a rate of 𝒪(K^-1/4) . Moreover, we derive an optimal sample complexity of 𝒪(ϵ ^-2) to find an ϵ -Pareto stationary point. Finally, preliminary numerical experiments validate the competitiveness of the proposed method.
This paper investigates gradient-type methods for solving optimization problems on Riemannian manifolds. In this approach, we utilize a search direction based on the gradient direction and employ a general non-monotone line search scheme to determine the stepsize at each iteration. This scheme encompasses several well-established non-monotone line search methods. Through our analysis, we demonstrate that this method exhibits asymptotic convergence characteristics and iteration-complexity bounds comparable to those of traditional Euclidean gradient-type methods using a non-monotone line search. The analysis presented significantly extends the study of gradient-type methods with line search to the domain of Riemannian manifolds, thereby providing several other line search options aiming at a better cost per iteration. Numerical experiments are also presented to illustrate the practical behavior of the proposed framework.
In this work, we apply a Fourier approach to study the stability and accuracy effects of filtering on Runge–Kutta (RK) discontinuous Galerkin (DG) methods for scalar hyperbolic conservation laws. Both the standard RKDG method and its variant, the RKDG method with compact stencils (cRKDG), will be considered. We focus on the second-order schemes and apply the modal filters immediately after the DG operators at each RK stage. Using a Fourier-type analysis, we quantitatively assess the time step constraints for stability and accuracy of multiple filtered schemes. We show that the application of filters can significantly improve the Courant–Friedrichs–Lewy numbers while maintaining the optimal convergence rate. However, for problems containing sonic points, the convergence order of some filtered schemes may degrade. To address this issue, we apply filters only away from the sonic points and adopt a local time-stepping strategy for the cRKDG method to take advantage of the larger time steps enabled by the filters in these regions.
This review synthesizes recent advances in loss function designs for Physics-Informed Neural Networks (PINNs), a transformative approach to solving partial differential equations (PDEs) by embedding physical laws into deep learning frameworks. We begin by exploring the foundational role of loss functions in deep neural networks and their adaptation in PINNs to enforce PDE residuals, boundary conditions, and data constraints. The article systematically categorizes core loss components and early optimization strategies, including collocation point selection, which underpin effective PINN training. A comprehensive survey examines advanced loss function designs, including adaptive and self-adaptive weighting strategies, alternative formulations such as weak forms, energy-based, and gradient-enhanced methods, as well as uncertainty quantification and training enhancements like meta-learning, transfer learning, loss landscape engineering, regularization techniques, and optimization algorithms. We evaluated their impacts on convergence speed, predictive accuracy, and computational efficiency in applications such as fluid dynamics, solid mechanics, wave propagation, and geotechnical modeling. Challenges such as gradient imbalances, multi-scale dynamics, and high-dimensional scaling are critically analyzed, with emerging solutions like automated loss balancing, physics-aware regularization, and innovative architectures proposed to enhance robustness. A comparative analysis highlights performance trade-offs, while future directions emphasize dynamic optimization, hierarchical multi-fidelity losses, and scalable uncertainty quantification. This structured synthesis, drawing on developments up to 2025, equips researchers with insights to refine PINN methodologies, bridging data-driven and physics-based paradigms for transformative scientific computing.
This paper addresses large-scale composite optimization problems consisting of a finite-sum smooth (possibly nonconvex) loss function and a nonsmooth regularization term. By integrating the variance-reduction mechanism of Point-SAGA with the Douglas-Rachford (DR) splitting paradigm, we propose an efficient framework termed the Variance-Reduced Douglas-Rachford (VR-DR) method. Unlike existing stochastic schemes, VR-DR sequentially evaluates the proximal operators of a sampled loss component and the regularization term, consistent with the classical DR splitting framework. To enhance computational tractability, we further develop an inexact variant (iVR-DR), which allows subproblems to be solved approximately under a controlled inexactness criterion. Rigorous convergence analysis in expectation is provided for both VR-DR and iVR-DR. Specifically, we establish sublinear convergence rates with respect to the proximal gradient mapping when the regularization term is convex. In the fully nonconvex and nonsmooth regime, we prove the global convergence and establish convergence rates for the generated iterative sequences by leveraging the Kurdyka-Łojasiewicz framework. Numerical experiments on sparse logistic regression, sparse least squares, and sparse support vector machines demonstrate the superior efficiency of our methods compared to state-of-the-art solvers. Notably, while our theoretical analysis assumes smoothness, empirical results reveal that VR-DR remains highly effective for nonsmooth loss functions, significantly broadening its practical utility beyond its theoretical constraints.
In this paper, an efficient numerical algorithm for large-scale ill-posed linear inverse problems in image restoration is proposed. To enhance computational speed, we extend the structured fast iterative shrinkage-thresholding algorithm (sFISTA- l_2 ) to solve the corresponding l_1 -regularized minimization problem by leveraging two hidden structures. Firstly, the coefficient matrix is approximated as a sum of a minority of Kronecker products, which enables the subsequent optimization procedure to utilize much smaller-scale and computationally superior matrix-matrix multiplications. Secondly, fast matrix–vector multiplication algorithms can be employed at each iteration since all matrices in the Kronecker product approximation are structured matrices. Consequently, our proposed algorithm is referred to as structured FISTA for l_1 -regularized optimization model (sFISTA- l_1 ). The l_1 -regularized optimization model is non-smooth, which poses challenges in analyzing convergence results. However, this paper proves that sFISTA- l_1 exhibits similar convergence properties to sFISTA- l_2 and provides a proof approach applicable to any non-smooth convex regularization term. Finally, numerical experiments are conducted to validate the efficiency of sFISTA- l_1 .
We propose a second-order backward differentiation formula (BDF2) finite difference scheme for the sixth-order functionalized Cahn–Hilliard (FCH) equation with a logarithmic Flory–Huggins potential. The scheme is based on a novel convex-concave decomposition of the free energy, which enables us to establish the scheme’s unique solvability and to derive a modified unconditional energy-dissipation law. Using the singular nature of the logarithmic potential as the phase-field variable approaches the pure states ± 1 , we verify the positivity-preserving property of numerical solutions at a theoretical level. Next, we perform a rigorous optimal rate convergence analysis, obtaining error estimates in the norms l^∞ (0, T; H_h^-1) ∩ l^2(0, T; H_h^2) and l^∞ (0, T; L_h^2) ∩ l^2(0, T; H_h^3) under a mild linear refinement condition C_1 h ≤ t ≤ C_2 h . Numerical experiments are presented to validate the theoretical results. Convergence tests confirm the scheme’s second-order accuracy in both time and space, while the energy dissipation and mass conservation properties are numerically verified. Simulations of the pearling bifurcation demonstrate that the proposed BDF2 scheme outperforms BDF1 schemes in accuracy when capturing complex dynamic transitions.
We study the derivative-free composite optimization problem consisting of a stochastic objective function with no available gradient information and a deterministic regularization term. A zeroth-order stochastic proximal Newton-type algorithm with a new norm test for adaptive sample size selection is proposed. The norm test ensures that the subsampled search direction is a faithful approximation of the full-batch direction. Both iteration complexity and oracle complexity of the algorithm are established for nonconvex and strongly convex objectives, respectively. Numerical experiments show that the adaptive sampling strategy is practical and efficient.
We derive energy error estimates in a fully-discrete setting for the first-order wave equation in its Friedrichs formulation, using third- and fourth-order explicit Runge–Kutta (ERK3 and ERK4) schemes in time combined with hybrid high-order (HHO) and weak Galerkin (WG) methods in space. We establish the key properties to address the static coupling between the cell and the face unknowns inherent in hybrid methods. On simplicial meshes, we prove optimal convergence rates in time and in space for both ERK3 and ERK4 under the usual CFL condition. The two key arguments in the proof are the special interpolation operator devised for hybridizable discontinuous Galerkin (HDG) methods, and recent bounds on the operator norm of Taylor polynomials applied to the discrete differential operator. On general polytopal meshes, using different arguments, we prove optimal convergence rates in time and quasi-optimal in space for ERK3 under the usual CFL condition. Numerical experiments illustrate the theoretical findings.
This paper extends the Hu–Zhang element for linear elasticity to curved domains, preserving strong symmetry and H(div) -conformity. The non-polynomial structure of the curved Hu–Zhang element makes it difficult to analyze the stability, which is overcome by establishing a novel inf-sup condition. Optimal convergence rates are achieved for all variables except for the stress in the L^2 -norm. This suboptimality originates from the fact that the divergence space of the curved Hu–Zhang element is not contained in the discrete displacement space, which is improved by local p-enrichment on boundary elements. Some numerical experiments validate the theoretical results.
In this paper, we introduce a spectral/spectral element method based on domain mapping to solve the transmission eigenvalue problem on a class of complex geometries. Initially, for configurations that contain sector-shaped regions, we employ a domain mapping technique to transform the original problem into a standard two-dimensional fourth-order coupled system with singular variable coefficients on a rectangular domain. Subsequently, we derive a crucial and fundamental pole condition, introduce a class of non-uniformly weighted Sobolev spaces alongside their corresponding approximation spaces, and establish a Legendre spectral approximation for the mapped equation, furnishing theoretical error estimates for the numerical eigenvalues and their associated eigenfunctions. Moreover, for configurations incorporating regular polygonal and L-shaped domains, we propose a spectral element method grounded in a second-order mixed formulation, and substantiate the algorithm’s effectiveness and spectral accuracy through extensive numerical examples.
A symmetric bilinear form corresponding to the biharmonic operator is combined with a corresponding bilinear form for the Laplacian to create a temporally 4th order corrected leap-frog scheme for the wave equation. The boundary conditions are imposed weakly, which allows for handling an unaligned domain with a stabilised cut-element methodology. The approach requires only one mass matrix solve per time-step. We develop fully discrete cutHermite and cutDG methodologies using cubic basis functions and Cartesian meshes. In the DG case additional symmetric interior penalties are added to impose sufficient continuity, while the higher continuity of the Hermite finite elements suffices. The schemes are shown to be stable under CFL conditions similar to the corresponding un-cut and un-corrected cases, and to yield 4th order accurate solutions. We also discuss how the biharmonic weak form can be used on its own to solve for example a biharmonic equation.
This study focuses on the numerical modeling of wave propagation in gain media, a critical element in laser physics that amplifies electromagnetic fields through electron interactions. The gain process is represented by a four-level atomic model combining Maxwell’s equations with nonlinear Ordinary Differential Equations to describe the evolution of electronic populations. While the Finite Difference Time Domain method has traditionally dominated this domain, this work introduces a high-order Discontinuous Galerkin Time-Domain method, tailored for solving such complex systems in 3D. This method is implemented with a second-order Leap-Frog temporal scheme and includes approximations for nonlinear terms. Stability is proven via energy estimates, both for the continuous problem and its discrete counterpart. Numerical validation, carried out on a 3D cubic cavity using manufactured solutions, confirms the accuracy and convergence of the method, followed by a detailed investigation of a three-dimensional physical case. The work highlights the potential of Discontinuous Galerkin Time-Domain methods to enhance the precision and flexibility of numerical simulations in laser physics.
Stochastic reaction–diffusion equations play an important role in characterizing the coupled mechanisms among nonlinear reaction, spatial diffusion, and random perturbations. However, during long-time integration, the high-precision computation of the mean, variance, higher-order moments, probability density functions (PDFs), and cumulative distribution functions (CDFs) remains a challenging task. In this paper, we propose a time-splitting Fourier spectral method based on Wiener chaos expansion and Karhunen–Loève expansion for stochastic reaction–diffusion equations driven by additive noise, aiming to achieve efficient and stable uncertainty quantification over long-time integration. The computational accuracy, long-time stability, and efficiency of the proposed method are validated by two examples with analytical reference solutions, namely an OU-type stochastic benchmark and a linear stochastic heat equation. Numerical results show that the method accurately computes the mean, variance, and higher-order moments such as the third and fourth moments, and efficiently reconstructs PDFs and CDFs. Compared with the traditional Monte Carlo method, the proposed method achieves higher statistical accuracy and better probability distribution reconstruction under the same or lower computational cost over long-time integration. Furthermore, the framework is applied to one- and two-dimensional nonlinear stochastic reaction–diffusion equations to analyze the competition among noise, diffusion, and nonlinear reaction, revealing the statistical equilibrium characteristics and dynamic transition mechanisms of stochastic reaction–diffusion systems during long-time evolution.
Structure preserving exact splitting based semi-Lagrangian methods are proposed for a plasma hybrid model with kinetic ions and massless electrons. Two subsystems with mass, momentum, and energy conservation are obtained by a Poisson bracket-based splitting method. For the subsystem in which the distribution functions and the fields are coupled, the second order and reversible modified implicit mid-point rule is used in time with the specially designed mean velocity. The distribution functions are not involved in the iterations and are solved by exact splittings with only one dimensional advections, which makes the proposed schemes efficient. The cancellation problem is overcome by the numerical schemes constructed. Moreover, for the case with a periodic boundary condition, the magnetic field obtained is divergence free, mass, momentum, and energy are conserved. The methods can be extended to cases with multiple ion species.
In this paper, we analyze higher-order weighted and shifted Grünwald-Letnikov (WSGL) schemes for tempered subdiffusion problems with nonsingular and singular source terms. Building on the correction technique, we employ the resolvent estimates of the numerical schemes to demonstrate that conventional high-order WSGL schemes achieve only first-order convergence, regardless of the smoothness of data. Drawing on the insights from Jin et al. (SIAM J. Sci. Comput., 39(6) (2017), A3129-A3152) regarding correction terms, we introduce tailored corrections in the initial steps and integral-differential approach for singular source terms in time, then propose modified higher-order WSGL schemes for the tempered subdiffusion equation. Rigorous analysis shows that our corrected schemes preserve high-order accuracy, attaining optimal convergence orders of O(τ ^k) (k=2,3,4) for both smooth and nonsmooth data. Numerical experiments are provided to validate our theoretical findings.
In this paper, a compact Hermite weighted essentially non-oscillatory (CHWENO) scheme is proposed for nonlinear degenerate parabolic equations. The scheme is developed by integrating a compact central difference discretization with a nonlinear Hermite WENO (HWENO) methodology. Compared with the HWENO scheme presented in [3], the CHWENO scheme has two significant advantages: (1) High efficiency—Compared with the reference HWENO scheme, the proposed CHWENO scheme has the significant improvement because the derivative values of the solutions are obtained directly using a compact central difference scheme, and we do not need to solve the auxiliary derivative equations. Numerical results show that CHWENO schemes reduce CPU time by approximately 30
