This paper introduces a universal accelerated fractional-order gradient method (AFOGM) and analyzes its convergence from the perspective of dynamical systems. Under an appropriate regularity condition for non-convex objective functions, we show that AFOGM attains R-linear convergence, providing an explicit description of its convergence parameter region. Numerical experiments on standard benchmark problems further demonstrate that AFOGM, including the fractional-order Heavy Ball (FOHB) method and the fractional-order Nesterov’s accelerated gradient (FONAG), possesses a larger convergence domain than classical gradient descent methods. Finally, we employ AFOGM as an optimizer for deep neural networks, where it outperforms classical momentum-based methods on the MNIST and Fashion-MNIST datasets.
This article investigates an inverse source problem for a subdiffusion equation with multiple fractional derivatives. The goal is to recover a temporally varying source term with low regularity from distributed measurements. The main challenges, posed by the variable coefficients and the source’s weak regularity, are addressed by establishing new coercivity properties of the fractional derivatives. This key innovation allows us to frame the analysis similarly to elliptic equations, proving the well-posedness of the forward prob- lem via the Lax-Milgram theorem and deriving stability estimates for the inverse problem under low regularity assumptions. A stability estimate is provided, and the effectiveness of the approach is supported by numerical experiments.
The objective of the paper is to develop the quadratic discontinuous finite volume methods (DFVM) for solving elliptic equations with variable coefficients. The proposed numerical schemes are composed of the discontinuous Galerkin (DG) method with different interior penalty formulations (IIPG, NIPG, SIPG) and the finite volume element method which the trial function is discontinuous quadratic element function. Subsequently, with the specialized projection techniques, we built up a bridge between the bilinear form of DFVM and that of DG method, which simplifies the analysis and proves the optimal error estimate of the schemes in the broken $H^1$ norm. Finally, we provide numerical simulations to validate the theoretical findings.
In this paper, we introduce novel high-dimensional delayed stochastic Navier-Stokes models (HD-DSNSMs) characterized by multi-fractional Gaussian noise (mGn) and then establish sufficient conditions for the existence and uniqueness of solutions. Since obtaining exact solutions for these models is mostly impossible, this work aims to develop an efficient approach by applying an approximation of multi-fractional Brownian motion (mBm) and then using physics-informed deep neural networks to predict solutions for the HD-DSNSMs characterized by mGn. The deep neural network parameters are optimized by minimizing a mean-squared error loss function, which is derived from the discretization of the HDDSNSMs characterized by mGn via the standard Euler-Maruyama scheme. Optimization is carried out with the Adam algorithm, and the model employs the activation function sin. Four test problems are conducted to show the efficiency and accuracy of the method. The results demonstrate the ability of the method to extract solution patterns even when pressure and delay increase. This approach proves adaptable and remarkably efficient in solving the discussed models, as evidenced by the research outcome. Our results illustrate that the HD-DSNSMs characterized by mGn improve the classical DSNSMs.
We propose a two-step Gauss-Newton method (TS-GNM) for solving nonsmooth equations. At each iteration, the TS-GNM solves both a Gauss-Newton equation and an approximate Gauss-Newton equation. A second-order derivative-free line search strategy is designed to ensure the global convergence of TS-GNM. Under the nonsingularity condition and the strong semismoothness of the underlying function, we prove that the TS-GNM converges quadratically. Furthermore, we demonstrate that the TS-GNM achieves a cubic convergence rate when the generalized Jacobian is locally Lipschitz continuous at the solutions. Finally, we pay particular attention to the absolute value equation and present some numerical results.
In this paper, we study the non-global local minimizers of the generalized trust-region subproblem (GTR), $\min\{x^\top A_0 x + 2b_0^\top x \mid x^\top A_1 x + 2b_1^\top x + c_1 \leq 0\}$, and its equality-constrained version (GTRE), which will be candidates of the global minimizers of the nonconvex quadratically constrained quadratic programming when the hard-case happens. Specifically, if there exists $\mu_1 \in \mathbb{R}$ such that $A_0 + \mu_1 A_1 \succ 0$, we prove for GTR and GTRE that, when $A_1 \succeq 0$ (or $A_1 \preceq 0$) there may exist at most one non-global local minimizer, and when $A_1$ is indefinite there may exist at most two non-global local minimizers. Moreover, if there exists $\mu_1 \in \mathbb{R}$ such that $A_0 + \mu_1 A_1 \prec 0$, we prove also that GTR and GTRE may have at most one non-global local minimizer. All the above three upper bounds are tight, i.e., none of them can be improved again. In summary, the famous Martínez's result is successfully generalized from $A_1 \succeq 0$ to the case that $\mu_0 A_0 + \mu_1 A_1 \succ 0$ for some $\mu_0, \mu_1 \in \mathbb{R}$. Finally, an algorithm is proposed either to find all the non-global local minimizers of GTR and GTRE or to confirm their nonexistence in a tolerance. Preliminary numerical results demonstrate the effectiveness of the algorithm.
In this paper, we are concerned with strong convergence of the given mixed truncated Euler-Maruyama method for stochastic delay differential equations with concave diffusion coefficients. We prove that, when the drift term satisfies some suitable local Lipschitz and one-sided Lipschitz conditions and the diffusion term satisfies some concave condition, the mixed truncated Euler-Maruyama method strongly converges to the exact solution in the $L^p$ sense. One example is presented to interpret the theory.
This work addresses the inverse problem of recovering sparse initial data in spatial fractional parabolic equations. The associated solution operator for initial-boundary value problem is continuous and compact, implying severe ill-posedness. To overcome this, an $ℓ_1$-regularized formulation is considered. The misfit functional is shown to be differentiable and strictly convex, and the well-posedness of the regularized problem is established, including existence, uniqueness, stability, and convergence. A numerical algorithm is proposed and implemented, incorporating Nesterov’s accelerated algorithm to enhance efficiency. Numerical experiments in both one and two spatial dimensions confirm the feasibility and accuracy of the proposed approach.
The logarithm of the determinant of Hermitian positive semi-definite matrices arises in numerous contexts in statistic, machine learning, and information and communication engineering. In this paper, based on the logarithm of determinant, we propose the log-determinant optimization model from the Riemannian optimization viewpoint to compute the dominant eigenvalues and associated eigenvectors of a Hermitian positive semi-definite matrix. The Riemannian trust-region method is employed to solve this optimization problem. Experimental results are reported to show the efficiency of the proposed method.
In this paper, we present two structure-preserving numerical schemes for the Landau-Lifshitz equation by combining the exponential scalar auxiliary variable method with the projection method. These schemes preserve both the length constraint and the modified energy dissipation law, ensuring numerical stability and accuracy. Moreover, they are particularly well-suited for studying the Landau-Lifshitz equation with higher-order energy terms, which have often been overlooked in earlier studies but have a significant impact on the stability, dynamics, and thermal behavior of magnetic skyrmions. We establish the unique solvability and energy stability of the schemes, and provide a rigorous error analysis. Numerical experiments are conducted to demonstrate the accuracy and effectiveness of the proposed schemes.
In this paper, we consider a class of three-composite nonconvex optimization problems, in which the nonsmooth function is further composed with a linear operator. This problem has many applications such as sparse signal recovery, image processing and machine learning. Based on the conjugate duality theory, we present an accelerated preconditioned primal-dual gradient algorithm for this problem. Compared with the existing algorithms, our algorithm only needs to calculate the proximal mapping of the conjugate function h & lowast; which is always convex and lower semicontinuous and it does not need to calculate the proximal mapping of nonconvex functions. This may significantly reduce the computation load. We prove that the sequence generated by the proposed algorithm globally converges to a critical point when the function satisfies the Kurdyka- Lojasiewicz property. We also obtain the convergence rate of the proposed algorithm. Finally, numerical results on sparse signal recovery and image processing illustrate the efficiency and competitiveness of the proposed algorithm.
In this paper, we consider the sparse distributed control problem constrained by a random elliptic equation, which we reformulate as a nonsmooth stochastic optimization problem in Hilbert space. By incorporating the advantages of the stochastic approximation approach and the alternating direction method of multipliers (ADMM), we propose a stochastic ADMM algorithm. This method decouples the stochasticity arising from the random equation constraint from the nonsmoothness of the control objective, allowing them to be tackled separately within the iterations. We introduce stochastic gradients and develop a proximal linearization technique for the stochastic subproblem, allowing each subproblem to admit a closed-form solution. The convergence and a high-probability bound of the proposed method are analyzed for the model problem. Numerical results demonstrate the effectiveness and efficiency of our method.
The high-resolution ordinary differential equation (ODE) framework has proven to be particularly effective for analyzing Nesterov's accelerated gradient (NAG) method and its proximal counterpart, the faster iterative shrinkage-thresholding algorithms (FISTA). However, the theoretical framework remains incomplete, as the underdamped regime (r < 2) has not yet been fully incorporated. In this paper, we extend the high-resolution ODE framework to fill this gap. By introducing a temporal scaling t (r+1/3) (or k r+1/3 in the discrete setting into the mixed term, we construct new Lyapunov functions specifically tailored for the underdamped case. Notably, when r = 2, our construction reduces exactly to the classical Lyapunov functions known from prior analyses. The proposed approach not only characterizes the convergence rate of the minimal squared gradient norm, but also recovers the objective-value convergence rate derived from the low-resolution ODE framework. Moreover, the convergence rates obtained for the underdamped regime depend continuously with respect to the damping parameter r. Finally, we demonstrate that the high-resolution ODE continues to faithfully capture the convergence behavior of NAG even in the critically damped regime r = -1, whereas the low-resolution ODE degenerates into the conservative Newtonian system and fails to describe the dissipative dynamics. In contrast, the high-resolution ODE framework consistently characterizes the convergence rates in a manner coherent with those obtained for the underdamped case as r = -1. r+1 3 (or k r+1 3 in the
The augmented Lagrangian method (ALM), firstly proposed in 1969, remains a vital framework in large-scale constrained optimization. This paper addresses a linearly constrained composite convex minimization problem and presents a general proximal ALM that incorporates both Nesterov acceleration and relaxed acceleration, while enjoying a proximal-indefinite term. Under mild assumptions without requiring prior knowledge of the objective function's strong convexity modulus, we establish the global convergence of the proposed method and derive an O(1/k2) nonergodic convergence rate for the Lagrangian residual, the objective gap, and the constraint violation, where k denotes the iteration number. Numerical experiments on testing large-scale sparse signal reconstruction tasks demonstrate the method's superior performance against several well-established methods.
This study presents a novel hybrid approach for addressing incompressible stationary natural convection problem, incorporating a parallel technique to enhance computational efficiency. Inspired by the traditional two-level method [He and Wang, Comput. Methods Appl. Mech. Engrg., 197 (2008)] and the two-step approach [Wu et al., Int. J. Heat Mass Transfer, 101 (2016)], both characterized by their iterative and corrective processes, we endeavor to alleviate the computational burden associated with the iterative process. Building upon these methods, our novel hybrid method involves two primary steps: initially solving the original problem using the finite element pair P1b - P1 - P1 on a coarse mesh, followed by resolving the linearized equations using the higher-order pair P2 - P1 - P2 on a fine mesh. While the first step employs iterative techniques, the second step entails directly solving a linearized problem. This novel approach can save lots of computational time in the iterative step compared to the traditional methods. Moreover, leveraging domain decomposition techniques, we implement a parallel strategy to further accelerate computations. Finally, we conduct several numerical examples to validate the efficiency of the proposed algorithms. The numerical results demonstrate optimal convergence rates relative error conditions. Furthermore, the numerical simulations on the two obstacles algorithms.
This paper presents a high-order multiple-invariants preserving numerical scheme for the modified Euler-Poincare equations with two components. It is shown that the scheme preserves at least three invariants: mass, momentum and energy. In contrast, the previous schemes usually keep only one or two. Meanwhile, the scheme achieves high order accuracy in spatial direction as well as second-order in time, which will be proved rigorously in this paper. The key to the present scheme aims at the construction of a bi-variate function and utilization of a special time discretization. Numerical tests are given to verify the theoretical findings.
The purpose of this paper is to investigate the superconvergence of collocation methods for fractional integro-differential equations (FIDEs) with weakly singular kernels and Caputo derivative of order 0 < alpha < 1. First, the initial value problem of FIDEs is reformulated as a weakly singular Volterra integral equation (VIE), and the existence, uniqueness, and regularity of the exact solution for the original FIDE are obtained with the help of the resolvent theory of VIEs, and it is shown that the singularity of the exact solution is governed by the Caputo derivative, not the weakly singular kernel. Next, the piecewise polynomial collocation method is employed to solve the reformulated VIE numerically, and the optimal convergence order of the collocation solution is obtained on graded meshes. In order to improve the numerical accuracy, two types of postprocessing techniques are used - one is the classical iterated technique for VIEs and another one is the interpolation postprocessing technique. The superconvergence is thoroughly investigated and the optimal superconvergence orders are obtained for both of these two postprocessing techniques. Compared to the classical iterated collocation method, the interpolation postprocessing method has a lower calculation cost. The theoretical results are illustrated by numerical experiments.
In this research, we employ a quasi-boundary-value method to address a class of nonlinear Riesz-Feller space fractional backward heat conduction problem, which are known to be severely ill-posed. A rigorous theoretical framework is established to verify the existence, uniqueness, and stability of the regularized solution. Furthermore, both a priori and a posteriori convergence estimates are derived under an a priori bound assumption on the exact solution, providing quantitative insight into the accuracy of the regularized approximation. For the numerical implementation, a finite difference scheme is combined with a fixed-point iteration, thereby illustrating the effectiveness of the proposed method. The results confirm that the proposed approach is not only theoretically sound but also practically effective for solving a class of nonlinear Riesz-Feller spatial fractional backward heat conduction problem.
In this paper, we introduce a class of stochastic Hopfield neural network with cross-diffusion (SHNNCD), and for the first time, investigate the existence of the ergodic stationary distribution for SHNNCD under time-varying modified Markov switching. In terms of the internal structure, we incorporate cross-diffusion to fully account for the coupling effects between different neuronal states. In terms of the external environment, compared with traditional Markov switching, we consider a class of Markov switching modified by time-varying functions, including periodic, decaying, and random types. Based on these innovations in both internal structure and external environment, we construct a global Lyapunov function for SHNNCD using graph theory under the consideration of cross-diffusion, and derive the criterion for the existence of the ergodic stationary distribution of SHNNCD. Finally, we verify the validity of our theoretical results through numerical simulations.
We introduce the statistical Romberg extrapolation method, a novel acceleration technique for discretizing forward backward stochastic differential equations. Compared to the classical Crank-Nicolson scheme, our single-step method achieves third-order convergence, which is significantly higher than the second-order convergence exhibited by the classical Crank-Nicolson scheme. Precise error estimates are derived, rigorously establishing this third-order convergence rate. Numerical experiments confirm the analysis and demonstrate a substantial improvement in accuracy.
