In this paper, we present a proximal gradient method with a new non-monotone line search for solving non-convex and non-smooth composite optimization problems. The method introduces a random term to determine whether the algorithm directly accepts the Barzilai-Borwein stepsize. Our approach uses a line-form backtracking line search, which ensures that the proximal operator is computed only once per iteration, offering an advantage over existing mainstream proximal gradient methods. Under mild conditions that do not require gradient Lipschitz continuity, we establish the subsequence convergence of the proposed method. Preliminary experiments show that our method outperforms existing non-monotone line search methods in terms of performance and efficiency.
This paper studies the structure of the core in many-to-one assignment games, where firms with limited capacity hire workers in a transferable utility framework. While it is well-established that the core of these games is always non-empty and contains firm-optimal and worker-optimal outcomes, relatively little is known about its complete geometric structure. In particular, regarding the characterization and enumeration of its extreme points. In this paper, we provide a graph-theoretic criterion for core vertices: a salary vector is a vertex of the core if and only if the base graph of its associated tight digraph is connected. We also provide a necessary and sufficient condition for each side-optimal allocation in terms of the tight digraph. Based on this characterization, we develop a lexicographic procedure that generates all core vertices as they are supported by a max-min salary vector, where workers sequentially optimize their payoffs with an indication of whether the worker in this position maximizes or minimizes his/her salary.
In this paper, we discuss a class of nonconvex-nonsmooth multi-block optimization with equality and closed convex set constraints, and the aim is to propose a novel fully symmetric Peaceman–Rachford (PR) regularization hybrid distributed method (HDM). The method is based on the ideas of the PR splitting and sequential quadratic programming (SQP) methods. It incorporates the Armijo line search technique and a sequential updating scheme for the Lagrange multiplier (dual variable). To ensure the solvability of nonsmooth subproblems, we add a proximal regularization term to each of them. This guarantees the existence and uniqueness of solutions, while improving numerical stability. For smooth subproblems, we use their quadratic approximations to enhance efficiency. The search direction corresponding to the smooth sub-objective is generated by solving a small-scale quadratic programming (QP), and then takes the augmented Lagrangian function (ALF) as the merit function to perform the Armijo line search in the sequential order. As a result, the new iteration of the associated primal block variable is yielded, respectively. Once a primal block variable is updated, the method immediately updates the dual variable with the latest iteration information, called it a fully symmetric Peaceman–Rachford distributed method in this sense. Under mild regularity assumptions, the global convergence and iteration complexity are established, and then the Maratos effect can be also overcome. Furthermore, by additionally imposing the condition that the merit function satisfies the Kurdyka–Łojasiewicz (KL) property and the gradients of the smooth block-objectives are locally Lipschitz continuous, the strong convergence and the convergence rate of the proposed method are established. A large number of compared numerical experiments on medium-to-large-scale problems show the effectiveness and robustness of the proposed method.
Extending Bayesian optimization to solve high-dimensional optimization problems is a meaningful but challenging task. As the dimension of the problem increases, it becomes exponentially more difficult to find the global optimum of the acquisition function, which often worsens the performance of Bayesian optimization in high-dimensional spaces. In this work, we propose a simple and efficient approach to tackle this problem. Instead of optimizing the acquisition function in the original high-dimensional space, we propose to use the block coordinate decent approach to optimize it block by block. The block size is initialized as the dimension of the problem, and deceases adaptively according to whether a better solution is found. The adaptive approach drives the search from global exploration to local exploitation gradually. In addition, we extend our approach to parallel evaluations in this work to further accelerate the Bayesian optimization process on high-dimensional problems. Extensive experiments have been conducted to verify the effectiveness of our proposed block coordinate decent approach. The results show our proposed algorithm can significantly improve the performance of Bayesian optimization on high-dimensional problems. Also, our algorithm shows very competitive optimization efficiency compared with six state-of-the-art high-dimensional Bayesian optimization algorithms. The codes of our proposed algorithm are available at https://github.com/zhandawei/Block_Coordinate_Descent_Bayesian_Optimization.
A Kantorovich-like optimal transport problem is formulated by seeking to minimize the Choquet integral of a given cost function with respect to the α -mixture of joint belief functions with given marginals, and their dual plausibility functions. This extension of the classical probabilistic problem allows to deal with misspecified marginal distributions by relying on a parameter α∈ [0,1] that acts like a pessimism index. We show that the particular subcase given by a marginal belief function and a marginal probability measure can be used to model a game under ambiguity, through the definition of the Stackelberg-Cournot-Nash equilibrium with Dempster-Shafer uncertainty and α -maxmin preferences. An algorithm is provided for approximating an equilibrium based on a suitable entropic formulation of the defined optimal transport problem. Finally, a distinguished application is found in a market with a finite set of types of investors having misspecified distribution. Investors’ choices result in a purchase portfolio on a finite set of assets which, in turn, depends on asset prices fixed by the market maker and the interactions among investors.
We propose an approach based on quadratic approximations for solving general Mixed-Integer Nonlinear Programming (MINLP) problems. Specifically, our approach entails the global approximation of the epigraphs of constraint functions by means of paraboloids, which are polynomials of degree two with univariate quadratic terms, and relies on a Lipschitz property only. These approximations are then integrated into the original problem. To this end, we introduce a novel approach to compute globally valid epigraph approximations by paraboloids via a Mixed-Integer Linear Programming (MIP) model. We emphasize the possibility of performing such approximations a-priori and providing them in form of a lookup table, and then present several ways of leveraging the approximations to tackle the original problem. We provide the necessary theoretical background and conduct computational experiments on instances of the MINLPLib. As a result, this approach significantly accelerates the solution process of MINLP problems, particularly those involving many trigonometric or few exponential functions. In general, we highlight that the proposed technique is able to exploit advances in Mixed-Integer Quadratically-Constrained Programming (MIQCP) to solve MINLP problems.
Abstract We consider the difference of convex (DC) optimization problem subject to box-constraints. Utilizing $$\varepsilon $$ ε -subdifferentials of the DC components of the objective function, we develop a new method for finding global solutions to this problem. This method is based on the combination of a local search method and a special procedure for escaping from nonglobal solutions. The method terminates when the solution found by a local search method cannot be improved anymore. The escaping procedure is designed using subsets of the $$\varepsilon $$ ε -subdifferentials of DC components. We compute the deviation between these subsets and determine $$\varepsilon $$ ε -subgradients providing this deviation. Using these $$\varepsilon $$ ε -subgradients we formulate a subproblem with a convex objective function. The solution to this subproblem is used as a starting point for a local search. We study the convergence of the conceptual version of the proposed method and discuss its implementation. A large number of academic test problems are used to show that the method requires a reasonable computational effort to find higher quality solutions than the four local methods for DC optimization. In addition, we apply the new method to find global solutions to DC optimization problems and compare its performance with that of two benchmark global optimization solvers.
Nonsmooth and nonconvex optimization problems are central to many machine learning applications, such as matrix factorization, tensor decomposition, and deep learning. Existing PALM-type algorithms and their inertial variants have shown practical success, yet they remain limited by restrictive parameter conditions and incomplete theoretical guarantees. In this paper, we propose an improved inertial stochastic proximal alternating linearized minimization algorithm (IiSPALM). The method introduces a novel double-inertial mechanism applied both before and after each block update, while avoiding the rigidity of nonzero inertial parameters required. By combining this design with variance reduced stochastic gradient estimators, we establish the theoretical results: IiSPALM achieves an iteration complexity of 𝒪(ε ^-2) for finding an ε -stationary point, guarantees linear convergence under the Polyak–Łojasiewicz condition, and ensures global convergence via the Kurdyka–Łojasiewicz property. Numerical experiments on nonnegative sparse matrix decomposition, tensor CP decomposition, and proximal neural networks demonstrate that IiSPALM consistently outperforms state-of-the-art deterministic and stochastic PALM-type algorithms. These results confirm that IiSPALM bridges the theoretical and practical gaps left by previous methods, offering a flexible and efficient framework for large-scale nonconvex optimization. The code is available at https://github.com/nothing2wang/IiSPALM .
This study investigates the problem of achieving International Environmental Agreements (IEAs) by focusing on specific normative properties. We model a world comprising n asymmetric countries. Each country produces goods and services, thereby generating private welfare and negative externalities affecting all countries. Based on a cost-benefit analysis, each state determines its production level and decides voluntarily on its participation in a coalition. Our analysis of the standard static non-cooperative game-theoretic model of coalition formation demonstrates that, under the resulting economic scenario, neither per-capita nor average envy-freeness properties are guaranteed. To foster more equitable cooperation, we define new coalition-formation rules wherein coalesced players maximize the joint utility function subject to constraints that prevent envy. By examining the particular case of a two-country world, we show that, by imposing a suitable aggregate emissions cap, the resulting economic scenario induces a balance of production among agents, thereby generating social equity. The analysis is extended through numerical simulations for a ten-country setting. The results indicate that, while self-enforcing agreements are typically limited to small coalitions, the introduction of equity considerations within the coalition structure significantly increases the number of stable coalitions compared to those obtained in the standard setting.
We present a unified framework of exact continuous formulations for a wide range of classical graph optimization problems, including the minimum (double) dominating set, vertex cover, maximum independent set, clique, s-plex, and max-cut, by leveraging the probabilistic method, and introduce several novel weighted extensions. The approach yields multilinear polynomial programs over the unit hypercube whose optimal values coincide with the corresponding combinatorial invariants. To assess practical performance, we conduct numerical experiments using a modern global solver (Gurobi) and five state-of-the-art local solvers (CONOPT, IPOPT, KNITRO, LOQO, SNOPT). We find that high-degree multilinear objectives can severely impede convergence, but that systematic degree reduction via Boole’s inequality drastically improves solution quality and robustness. The obtained results are promising and encourage a more detailed investigation of continuous formulations based on the probabilistic method in the context of solving discrete optimization problems.
This work proposes a novel primal-dual dynamical model for affinely constrained convex optimization and presents a unified convergence analysis for a class of new accelerated primal-dual methods. In the continuous level, exponential decay of a novel Lyapunov function is established and in the discrete level, implicit, semi-implicit and explicit numerical discretizations for the continuous model are considered sequentially and lead to several accelerated primal-dual methods. To weaken the strong convexity assumption on the objective, we adopt the idea of quadratic penalty. Special structures of the corresponding subproblems are utilized to develop efficient inner solvers. In addition, nonergodic sublinear and linear rates in terms of the primal-dual gap, the objective residual and the feasibility violation are proved via a discrete Lyapunov function. Numerical results are provided to verify the practical performances of the proposed methods.
We consider problems with multiple conflicting objective functions and multiple decision makers and concentrate on interactive methods, where the preferences of the decision makers are iteratively incorporated in the solution process. We propose a novel trade-off-free interactive multiobjective optimization method NAUTILI for group decision making. In NAUTILI, the solution process starts from an inferior solution. Then, each decision maker individually interacts with NAUTILI via specialized visualizations in a graphical user interface to provide their preferences. NAUTILI determines the direction of simultaneous improvements for the whole group. Thus, in each iteration, all decision makers gain in all objective functions until reaching the most preferred solution by the group. This solution is so-called Pareto optimal. The method can be seen as a win-win approach, as in each iteration, all the objective functions can be improved simultaneously for all decision makers. As a proof of concept, we demonstrate the potential and benefits of NAUTILI with a forest management problem involving three decision makers. We also asked the decision makers to answer a questionnaire about their satisfaction with the final solution, the method’s user-friendliness, and how they felt that their preferences were reflected in the solution process. The findings support the applicability of the method. This supports the potential of applying an interactive and trade-off-free method to group decision making in multiobjective optimization.
In this paper we consider nonconvex diagonal Quadratically Constrained Quadratic Programming (QCQP) problems, where at least one constraint is strictly convex. A lower bound for such problems can be computed through a Lagrangian relaxation, where all constraints, except the strictly convex constraint, are moved into the objective function, suitably weighted by a vector of Lagrange multipliers. The search for the best (largest) Lagrangian bound leads to the definition of the dual Lagrangian problem. The aim of this paper is to provide sufficient as well as necessary and sufficient conditions under which exactness of the dual Lagrangian bound (i.e., the equivalence of its optimal value with the optimal value of the original nonconvex problem) is guaranteed. Some of these conditions are based on the solution of linear feasibility problems defined over the space of the Lagrange multipliers, some others on the solution of convex problems over the same space. We specialize the results proved for general diagonal QCQPs to the special case where the constraints are linear, ball and reverse ball constraints, i.e., the Hessian matrices of the constraint quadratic functions are null matrices (linear constraints), identity matrices (ball constraints), and the opposite of identity matrices (reverse ball constraints). We show that in this special case the conditions can be evaluated much more efficiently. Finally, we perform experiments over random instances to identify relevant factors which affect both exactness of the dual Lagrangian bound and the ability of detecting such exactness through the proposed sufficient conditions.
This paper studies mathematical programs with second-order cone complementarity constraints (SOCMPCCs), which generalize classical mathematical programs with complementarity constraints by incorporating second-order cone structures. SOCMPCCs present substantial theoretical and computational challenges because standard constraint qualifications, such as Robinson’s condition, are violated at every feasible point. This prevents the direct application of classical nonlinear programming theories and algorithms. To address these difficulties, we develop a class of smoothing methods that approximate the original SOCMPCC using suitably constructed smoothing functions. We prove that any sequence of stationary points of approximate problems converges to a C-stationary point of the original SOCMPCC under a newly introduced constraint qualification, named SOCMPCC-WLICQ. This condition is proven to be strictly weaker than the widely assumed SOCMPCC-LICQ, yet strictly stronger than the SOCMPCC nondegeneracy condition. Furthermore, we demonstrate that the resulting C-stationary point can be strengthened to an S-stationary point when an additional strict complementarity condition is satisfied. Numerical experiments further verify theoretical results and illustrate the effectiveness of the proposed smoothing methods, as evidenced by their improved performance on several test problems.
Typically, probability distributions that generate uncertain parameters cannot be measured exactly in practice. As a remedy, distributional robustness determines optimized decisions that are protected in a robust fashion against all probability distributions in some appropriately chosen ambiguity set. In this work, we consider robust joint chance-constrained optimization problems and focus on discrete probability distributions. Many methods for this kind of problems study convex or even linear constraint functions. In contrast, we introduce a practically efficient scenario-based bundle method without convexity assumptions on the constraint functions. We start by deriving an approximation problem to the original robust chance-constrained version by using smoothing and penalization techniques that build on our former work on chance-constrained optimization. Our convergence results with respect to the smoothing approximation and well-known results for penalty approximations suggest replacing the original problem with the approximation problem for large smoothing and penalty parameters. Our scenario-based bundle method starts by solving the approximation problem with a bundle method, and then uses the bundle solution to decide which scenarios to include in a scenario-expanded formulation. This formulation is a standard nonlinear optimization problem. Our approach is guaranteed to find feasible solutions. Furthermore, in the numerical experiments on real-world gas transport problems with uncertain demands, we mostly find globally optimal solutions. Comparing these results to the classical robust reformulations for ambiguity sets consisting of confidence intervals and Wasserstein balls, we observe that the scenario-based bundle method typically outperforms solving the classical reformulation directly.
The preference incorporated space (IOPIS) paradigm in interactive multiobjective optimization introduces a transformation or mapping of the objective space to a space where the coordinates are the values of scalarization functions. The scalarization functions include preference information provided by a decision maker. Therefore, IOPIS provides new possibilities for solving multiobjective optimization problems. In this paper, we investigate the effects of the transformation on the landscapes of multiobjective optimization problems by altering the landscape and the distribution of locally Pareto optimal solutions. The analysis includes both theoretical proofs and illustrations showcasing the landscapes of multiobjective optimization problems. Theoretical results establish that the transformation using achievement scalarizing functions does not introduce any new locally Pareto optimal solutions, i.e., if a solution is not locally Pareto optimal for the original problem, it cannot become locally Pareto optimal after the transformation. Moreover, we derive conditions under which local Pareto optima in the original problem are preserved and derive a geometrical description of the region where local optima can be found. Visualizations illustrate the theoretical findings for a number of examples. Besides its advantages in interactive multiobjective optimization, the IOPIS transformation may also help “smooth out” local optima in conventional a posteriori methods while retaining global optima.
Nonconvex and nonsmooth composite optimization problems with linear constraints have attracted widespread attention in artificial intelligence and computer science due to their extensive applications in machine learning. Variance-reduced stochastic ADMM algorithms are extensively employed to address these problems, with most existing methods relying on the classical SVRG double-loop framework. In this study, we introduce the SVRRM-ADMM algorithm, which leverages a novel stochastic variance-reduced recursive momentum (SVRRM) estimator constructed on the loopless-SVRG framework and integrates it with the Stochastic ADMM (SADMM). The proposed algorithm facilitates implementation, requires fewer tuning parameters, and retains equivalent theoretical properties. We demonstrate that SVRRM-ADMM converges to a stationary solution without assuming bounded variance. Moreover, we extend the SVRRM-ADMM algorithm by incorporating acceleration techniques to yield the ASVRRM-ADMM. We establish that both SVRRM-ADMM and ASVRRM-ADMM achieve a worst-case convergence rate of O(1/T), where T denotes the number of iterations. Under the additional Kurdyka-Lojasiewicz (KL) property assumption, we further show that the sequences generated by both algorithms have finite expected length and attain different convergence rates determined by the KL exponent. Finally, numerical experiments validate the effectiveness of the proposed algorithms.
In recent years, various subspace algorithms have been developed to handle large-scale optimization problems. Although existing subspace Newton methods require fewer iterations to converge in practice, the matrix operations and full gradient computation are bottlenecks when dealing with large-scale problems. We propose a subspace quasi-Newton method that is restricted to a deterministic subspace together with a subspace gradient based on random matrix theory. Our method does not require full gradients, let alone Hessian matrices. Yet, it achieves the same order of worst-case iteration complexity in expectation for both convex and nonconvex cases as existing subspace methods. In numerical experiments, we confirm the superiority of our algorithm in terms of computation time.
This paper addresses the split feasibility problem with multiple output sets in Hilbert spaces, wherein the feasibility sets are represented as sublevel sets of convex functions. In contrast to existing relaxed projection algorithms wherein half-spaces are constructed in distinct Hilbert spaces, the proposed approach constructs all half-spaces uniformly within a single Hilbert space. Based upon this conceptual framework, several relaxed projection algorithms are developed. Under standard conditions, the convergence of these algorithms is rigorously established. Numerical experiments demonstrate the effectiveness of the proposed algorithms and provide comparisons with existing relaxed projection methods.
