Leveraging Tucker low-rank decomposition for dimensionality reduction and over-fitting mitigation, this study investigates stochastic tensor Tucker optimization problems subject to deterministic constraints, with an emphasis on designing a Stochastic Proximal Alternating Direction Method of Multipliers (SPADMM). Instead of treating the underlying polynomial structures as general smooth nonlinear functions, we introduce block conservative functions based on the tensor Tucker decomposition, and further analyse the convergence rate and high probability performance for the proposed stochastic algorithm. Both numerical tests and background/foreground separation experiments demonstrate that our approach is promising in time efficiency and solution accuracy.
In this paper, we develop a modified inertial triple self-adaptive algorithm that integrates the inertial method, the subgradient extragradient method, and the hybrid steepest descent method for solving bilevel variational inequality problems, where the upper-level problem involves a strongly monotone variational inequality problem and the lower-level problem comprises a quasimonotone variational inequality problem with split variational inequalities and fixed-point constraints. A distinctive feature of our approach is a triple self-adaptive step-size mechanism that dynamically adjusts parameters without requiring prior estimation of operator norms or Lipschitz constants. We rigorously establish the strong convergence theory for the algorithm under the bilevel framework. Experimental results on two numerical examples and signal recovery applications, including the modified LASSO problem, demonstrate that the proposed algorithm achieves superior performance compared to existing methods in terms of both computational efficiency and practical applicability.
The heavy-ball (HB) method is a classical momentum scheme for large-scale optimization, with well-known fast local behaviour under smoothness and strong convexity in Euclidean spaces. We develop a Riemannian heavy-ball method on manifolds equipped with a retraction and an associated vector transport, allowing practical implementations beyond geodesic updates. Our analysis does not rely on strong convexity, for general smooth (possibly nonconvex) objectives, we establish global convergence in the sense that the update directions are square summable and the Riemannian gradient norm vanishes along a subsequence. Under the Riemannian Polyak-& Lstrok;ojasiewicz (PL) inequality, a gradient-dominance condition strictly weaker than strong convexity, we further prove a global linear convergence rate in function values. These guarantees also specialize to the Euclidean setting and yield nonconvex global results for HB under the PL condition. Numerical experiments implemented with Manopt illustrate the practical performance of the method on manifold-structured tasks, including sparse PCA via Stiefel-type formulations and sphere-constrained signal denoising.
We propose a regularized proximal Newton algorithm with partial Hessian information for minimizing the sum of a twice continuously differentiable function and a separable convex (possibly nonsmooth) function. We prove that every accumulation point of the iterates is a stationary point while allowing the inner minimization to be solved inexactly. Moreover, we prove that when the problem possesses the local H & ouml;lderian error bound property, the local convergence rate of the iterates is superlinear. The efficiency of the proposed algorithm is demonstrated by comparisons with several state-of-the-art algorithms on the & ell;1 -regularized problems.
The aim of this paper is to study a double inertial extragradient algorithm with double projection and prove a weak convergence theorem for finding a common solution of a variational inequality problem and a fixed point problem in a real Hilbert space. Furthermore, a strong convergence theorem of the proposed algorithm is established under very mild conditions. Finally, we give a numerical example to illustrate the feasibility and effectiveness of our algorithm.
