Abstract. Local superlinear convergence of the semismooth Newton method usually necessitates assumptions on the uniform invertibility of the utilized, generalized Jacobian matrices, such as, e.g., BD- or CD-regularity. For certain composite-type problems and nonlinear programs (for which explicit representations of the generalized Jacobians of the associated stationarity equations are available), such regularity assumptions are closely connected to strong second-order sufficient conditions. However, general characterizations are not well understood. In this paper, we investigate a strong second-order sufficient condition ([Formula: see text]) for composite problems whose nonsmooth part has a generalized conic-quadratic second subderivative. We discuss the relationship between the [Formula: see text] and other second order-type conditions that involve the generalized Jacobians of the normal map. In particular, these two conditions are equivalent under certain structural assumptions on the generalized Jacobian matrix of the proximity operator. Leveraging second-order variational techniques and properties, we then verify that the introduced structural conditions hold for a broad class of [Formula: see text]-strictly decomposable functions. Finally, it is shown that the [Formula: see text] is further equivalent to the strong metric regularity of the subdifferential, the normal map, and the natural residual.
Abstract. This paper presents an algorithmic study and complexity analysis for solving distributionally robust multistage convex optimization (DR-MCO) problems. Our main contribution is a novel nonconsecutive dual dynamic programming (NDDP) algorithm which explores different stages in an adaptive fashion. In contrast with the usual consecutive dual dynamic programming algorithm, we show that NDDP reduces the subproblem complexity from quadratic to linear dependency on the number of stages. Two different DR-MCO examples are also presented to show the efficiency and effectiveness of the proposed NDDP algorithm.
Abstract. In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak [ USSR Comput. Math. Math. Phys., 9 (1969), pp. 14–29]. We study the convergence behavior of PolyakGD from two perspectives: tight worst-case analysis and universality across function classes. As our first main result, we establish the tightness of the known convergence rates of PolyakGD by explicitly constructing worst-case functions. In particular, we show that both the [Formula: see text] rate for smooth strongly convex functions and the [Formula: see text] rate for smooth convex functions are tight. Moreover, we theoretically show that PolyakGD automatically exploits floating-point errors to escape the worst-case behavior. Our second main result provides new convergence guarantees for PolyakGD under both Hölder smoothness and Hölder growth conditions. These findings show that the Polyak stepsize is universal, automatically adapting to various function classes without requiring prior knowledge of problem parameters.
Abstract. We analyze the consensus-based optimization algorithm with a consensus point rescaled by a fixed parameter [Formula: see text]. Under minimal assumptions on the objective function and the initial data, we establish its unconditional convergence to the global minimizer. Our results hold in the asymptotic regime where the time horizon [Formula: see text] and the inverse temperature [Formula: see text] are taken successively, providing a rigorous theoretical foundation for the algorithm’s global convergence. Furthermore, our findings extend to the case of multiple and nondiscrete set of minimizers.
Abstract. We discuss optimization problems over convex cones in which membership is difficult to verify directly. In the standard theory of duality, vectors in the dual cone [Formula: see text] are associated with separating hyperplanes and interpreted as certificates of nonmembership in the primal cone [Formula: see text]. Complementing this perspective, we develop easily verifiable certificates of membership in [Formula: see text] given by vectors in [Formula: see text]. Assuming that [Formula: see text] admits an efficiently computable logarithmically homogeneous self-concordant barrier, every vector in the interior of [Formula: see text] is associated with a full-dimensional cone of efficiently verifiable membership certificates. Consequently, rigorous certificates can be computed using numerical methods, including interior-point algorithms. The proposed framework is particularly well suited to optimization over low-dimensional linear images of higher-dimensional cones: We argue that these problems can be solved by optimizing directly over the (low-dimensional) dual cone, circumventing the customary lifting that introduces a large number of auxiliary variables. As an application, we derive a novel closed-form formula for computing exact primal feasible solutions from suitable dual feasible solutions; as the dual solutions approach optimality, the computed primal solutions do so as well. To illustrate the generality of our approach, we show that the new certification scheme is applicable to virtually every tractable subcone of nonnegative polynomials commonly used in polynomial optimization (such as sums of squares, SONC, SAGE, and SDSOS, among others), facilitating the computation of rigorous nonnegativity certificates using numerical algorithms.
