The goal of this article is to study necessary and sufficient conditions for the exactness of penalty functions and the existence of global saddle points of augmented Lagrangians for well-posed (in a suitable sense) constrained optimization problems in infinite dimensional spaces. To this end, we present a new version of extended well-posedness of a constrained optimization problem and analyze how it relates to the more well-known types of well-posedness, such as Tikhonov and Levitin–Polyak well-posedness. This new version of extended well-posedness allows one to extend many existing results on exact penalty functions and global saddle points of augmented Lagrangians from the finite dimensional to the infinite dimensional case. Such extensions provide first verifiable sufficient conditions for the exactness of penalty functions and the existence of global saddle points of augmented Lagrangians in the infinite dimensional case that do not rely on very restrictive and difficult to verify assumptions (nonlocal metric regularity of constraints, existence of nonlocal error bounds, the Palais–Smale condition, abstract properties of the perturbation function, etc.) that are typically used in the literature.

A hypodifferential is a compact family of affine mappings that defines a local max-type approximation of a nonsmooth convex function. We present a general theory of hypodifferentials of nonsmooth convex functions defined on a Banach space. In particular, we provide complete characterizations of hypodifferentiability and hypodifferentials of nonsmooth convex functions, derive calculus rules for hypodifferentials, and study the Lipschitz continuity/Lipschitz approximation property of hypodifferentials that can be viewed as a natural extension of the Lipschitz continuity of the gradient to the general nonsmooth setting. As an application of our theoretical results we study the rate of convergence of several versions of the method of hypodifferential descent for non- smooth convex optimization and present an accelerated version of this method having the faster rate of convergence O(1/k(2)).

In this two-part study, we develop a general theory of the so-called exact augmented Lagrangians for constrained optimization problems in Hilbert spaces. In contrast to traditional nonsmooth exact penalty functions, these augmented Lagrangians are continuously differentiable for smooth problems and do not suffer from the Maratos effect, which makes them especially appealing for applications in numerical optimization. Our aim is to present a detailed study of various theoretical properties of exact augmented Lagrangians and discuss several applications of these functions to constrained variational problems, problems with PDE constraints, and optimal control problems. The first paper is devoted to a theoretical analysis of an exact augmented Lagrangian for optimization problems in Hilbert spaces. We obtain several useful estimates of this augmented Lagrangian and its gradient, and present several types of sufficient conditions for KKT-points of a constrained problem corresponding to locally/globally optimal solutions to be local/global minimizers of the exact augmented Lagrangian.

Subdifferentials (in the sense of convex analysis) of matrix-valued functions defined on ℝ^d that are convex with respect to the Löwner partial order can have a complicated structure and might be very difficult to compute even in simple cases. The aim of this paper is to study subdifferential calculus for such functions and properties of their subdifferentials. We show that many standard results from convex analysis no longer hold true in the matrix-valued case. For example, in this case the subdifferential of the sum is not equal to the sum of subdifferentials, the Clarke subdifferential is not equal to the subdifferential in the sense of convex analysis, etc. Nonetheless, it is possible to provide simple rules for computing nonempty subsets of subdifferentials (in particular, individual subgradients) of convex matrix-valued functions in the general case and to completely describe subdifferentials of such functions defined on the real line. As a by-product of our analysis, we derive some interesting properties of convex matrix-valued functions, e.g. we show that if such function is nonsmooth, then its diagonal elements must be nonsmooth as well.

The goal of this note is to point out an erroneous formula for the generalized Hessian of the least squares associated with a system of linear inequalities, that was given in the paper 'A finite Newton method for classification' by Mangasarian (Optim. Methods Softw. 17 (2002), pp. 913-929) and reproduced multiple times in other publications. We also provide sufficient contiditions for the validity of Mangasarian's formula and show that Slater's condition guarantees that some particular elements from the set defined by Mangasarian belong to the generalized Hessian of the corresponding function.

A class of exact penalty-type local search methods for optimal control problems with nonsmooth cost functional, nonsmooth (but continuous) dynamics, and nonsmooth state and control constraints is presented, in which the the penalty parameter and several line search parameters are adaptively adjusted during the optimisation process. This class of methods is applicable to problems having a known DC (Difference-of-Convex functions) structure and in its core is based on the classical DCA method, combined with the steering exact penalty rules for updating the penalty parameter and an adaptive nonmonotone line search procedure. Under the assumption that all auxiliary subproblems are solved only approximately (that is, with finite precision), we prove the correctness of the proposed family of methods and present its detailed convergence analysis. The performance of several different versions of the method is illustrated by means of a numerical example, in which the methods are applied to a semi-academic optimal control problem with a nonsmooth nonconvex state constraint.

The second part of our study is devoted to a detailed convergence analysis of two extensions of the well-known DCA method for solving DC (Difference of Convex functions) optimization problems to the case of general cone constrained DC optimization problems. We study the global convergence of the DCA for cone constrained problems and present a comprehensive analysis of a version of the DCA utilizing exact penalty functions. In particular, we study the exactness property of the penalized convex subproblems and provide two types of sufficient conditions for the convergence of the exact penalty method to a feasible and critical point of a cone constrained DC optimization problem from an infeasible starting point. In the numerical section of this work, the exact penalty DCA is applied to the problem of computing compressed modes for variational problems and the sphere packing problem on Grassmannian.

The paper is devoted to a detailed analysis of nonlocal error bounds for nonconvex piecewise affine functions. We both improve some existing results on error bounds for such functions and present completely new necessary and/or sufficient conditions for a piecewise affine function to have an error bound on various types of bounded and unbounded sets. In particular, we show that any piecewise affine function has an error bound on an arbitrary bounded set and provide several types of easily verifiable sufficient conditions for such functions to have an error bound on unbounded sets. We also present general necessary and sufficient conditions for a piecewise affine function to have an error bound on a finite union of polyhedral sets (in particular, to have a global error bound), whose derivation reveals a structure of sublevel sets and recession functions of piecewise affine functions.

We propose and study a version of the DCA (Difference-of-Convex functions Algorithm) using the $\ell_1$ penalty function for solving nonsmooth DC optimization problems with nonsmooth DC equality and inequality constraints. The method employs an adaptive penalty updating strategy to improve its performance. This strategy is based on the so-called steering exact penalty methodology and relies on solving some auxiliary convex subproblems to determine a suitable value of the penalty parameter. We present a detailed convergence analysis of the method and illustrate its practical performance by applying the method to two nonsmooth discrete optimal control problem.

This two-part study is devoted to the analysis of the so-called exact augmented Lagrangians, introduced by Di Pillo and Grippo for finite dimensional optimization problems, in the case of optimization problems in Hilbert spaces. In the second part of our study we present applications of the general theory of exact augmented Lagrangians to several constrained variational problems and optimal control problems, including variational problems with additional constraints at the boundary, isoperimetric problems, problems with nonholonomic equality constraints (PDE constraints), and optimal control problems for linear evolution equations. We provide sufficient conditions for augmented Lagrangians for these problems to be globally/completely exact, that is, conditions under which a constrained variational problem/optimal control problem becomes equivalent to the problem of unconstrained minimization of the corresponding exact augmented Lagrangian in primal and dual variables simultaneously.

We present a simple and efficient acceleration technique for an arbitrary method for computing the Euclidean projection of a point onto a convex polytope, defined as the convex hull of a finite number of points, in the case when the number of points in the polytope is much greater than the dimension of the space. The technique consists in applying any given method to a "small" subpolytope of the original polytope and gradually shifting it, till the projection of the given point onto the subpolytope coincides with its projection onto the original polytope. The results of numerical experiments demonstrate the high efficiency of the proposed acceleration technique. In particular, they show that the reduction of computation time increases with an increase of the number of points in the polytope and is proportional to this number for some methods. In the second part of the paper, we also discuss a straightforward extension of the proposed acceleration technique to the case of arbitrary methods for computing the distance between two convex polytopes, defined as the convex hulls of finite sets of points.

This work is devoted to an analysis of exact penalty functions and optimality conditions for nonsmooth two-stage stochastic programming problems. To this end, we first study the co-/quasi-differentiability of the expectation of nonsmooth random integrands and obtain explicit formulae for its coand quasidifferential under some natural assumptions on the integrand. Then we analyse exact penalty functions for a variational reformulation of two-stage stochastic programming problems and obtain sufficient conditions for the global exactness of these functions with two different penalty terms. In the end of the paper, we combine our results on the co-/quasi-differentiability of the expectation of nonsmooth random integrands and exact penalty functions to derive optimality conditions for nonsmooth two-stage stochastic programming problems in terms of codifferentials.

We present a general theory of exact penalty functions with vectorial (multidimensional) penalty parameter for optimization problems in infinite dimensional spaces. In comparison with the scalar case, the use of vectorial penalty parameters provides much more flexibility, allows one to adaptively and independently take into account the violation of each constraint during optimization process, and often leads to a better overall performance of an optimization method using an exact penalty function. We obtain sufficient conditions for the local and global exactness of penalty functions with vectorial penalty parameters and study convergence of global exact penalty methods with several different penalty updating strategies. In particular, we present a new algorithmic approach to an analysis of the global exactness of penalty functions, which contains a novel characterisation of the global exactness property in terms of behaviour of sequences generated by certain optimization methods.

The paper is devoted to an analysis of optimality conditions for nonsmooth multidimensional problems of the calculus of variations with various types of constraints, such as additional constraints at the boundary and isoperimetric constraints. To derive optimality conditions, we study generalised concepts of differentiability of nonsmooth functions called codifferentiability and quasidifferentiability. Under some natural and easily verifiable assumptions we prove that a nonsmooth integral functional defined on the Sobolev space is continuously codifferentiable and compute its codifferential and quasidifferential. Then we apply general optimality conditions for nonsmooth optimisation problems in Banach spaces to obtain optimality conditions for nonsmooth problems of the calculus of variations. Through a series of simple examples we demonstrate that our optimality conditions are sometimes better than existing ones in terms of various subdifferentials, in the sense that our optimality conditions can detect the non-optimality of a given point, when subdifferential-based optimality conditions fail to disqualify this point as non-optimal.

We study several versions of the alternating direction method of multipliers (ADMM) for solving the convex problem of finding the distance between two ellipsoids and the nonconvex problem of finding the distance between the boundaries of two ellipsoids. In the convex case we present the ADMM with and without automatic penalty updates and demonstrate via numerical experiments on problems of various dimensions that our methods significantly outperform all other existing methods for finding the distance between ellipsoids. In the nonconvex case we propose a heuristic rule for updating the penalty parameter and a heuristic restarting procedure (a heuristic choice of a new starting for point for the second run of the algorithm). The restarting procedure was verified numerically with the use of a global method based on KKT optimality conditions. The results of numerical experiments on various test problems showed that this procedure always allows one to find a globally optimal solution in the nonconvex case. Furthermore, the numerical experiments also demonstrated that our version of the ADMM significantly outperforms existing methods for finding the distance between the boundaries of ellipsoids on problems of moderate and high dimensions.

New combined finite-differential versions of nonsmooth speed-gradient (SG) algorithms are proposed and examined. Sufficient conditions for stability and robustness of the closed loop system are established. In addition, new passivity definitions suitable for the nonsmooth setting are proposed and passivity of the system with nonsmooth SG algorithms is examined. The proposed finite-differential algorithms possess enhanced dynamic properties and provide extra flexibility for control system design. Particularly, they may operate under a broad uncertainty of plant parameters and disturbances, and they have improved convergence rate and robustness. An illustrative example of adaptive control of Duffing system demonstrates better performance of a combination of smooth and nonsmooth finite-differential SG-algorithms in comparison with performance of the smooth or nonsmooth algorithm.

The class of nonsmooth codifferentiable functions was introduced by professor V.F. Demyanov in the late 1980s. He also proposed a method for minimizing these functions called the method of codifferential descent (MCD). However, until now almost no theoretical results on the performance of this method on particular classes of nonsmooth optimization problems were known. In the first part of the paper, we study the performance of the method of codifferential descent on a class of nonsmooth convex functions satisfying some regularity assumptions, which in the smooth case are reduced to the Lipschitz continuity of the gradient. We prove that in this case the MCD has the iteration complexity bound O ( 1 / epsilon ) . In the second part of the paper we obtain new global optimality conditions for piecewise affine functions in terms of codifferentials. With the use of these conditions we propose a modification of the MCD for minimizing piecewise affine functions (called the method of global codifferential descent) that does not use line search, and discards those 'pieces' of the objective functions that are no longer useful for the optimization process. Then we prove that the MCD as well as its modification proposed in the article find a point of global minimum of a nonconvex piecewise affine function in a finite number of steps.

The problem of observer-based boundary control of the sine–Gordon model energy is first posed. A Luenberger-type observer for the sine–Gordon equation is analysed, and explicit bounds on the system parameters ensuring the exponential decay of the estimation error are obtained. With the use of this observer a speed-gradient output feedback for sine–Gordon model energy control is proposed. The achievement of the control goal is established under the assumption that system’s energy does not vanish in finite time. It is shown that the transient time in energy is close that the transient time in observation error, i.e. the closed-loop system has a reasonable performance. It is also shown that the conditions on the observer gain sufficient for the state estimation error convergence are “almost” necessary in the sense that violation of these conditions makes the state estimation error transients too long.

Summary The second part of our study is devoted to an analysis of the exactness of penalty functions for optimal control problems with terminal and pointwise state constraints. We demonstrate that with the use of the exact penalty function method one can reduce fixed‐endpoint problems for linear time‐varying systems and linear evolution equations with convex constraints on the control inputs to completely equivalent free‐endpoint optimal control problems, if the terminal state belongs to the relative interior of the reachable set. In the nonlinear case, we prove that a local reduction of fixed‐endpoint and variable‐endpoint problems to equivalent free‐endpoint ones is possible under the assumption that the linearized system is completely controllable, and point out some general properties of nonlinear systems under which a global reduction to equivalent free‐endpoint problems can be achieved. In the case of problems with pointwise state inequality constraints, we prove that such problems for linear time‐varying systems and linear evolution equations with convex state constraints can be reduced to equivalent problems without state constraints, provided one uses the L ∞ penalty term, and Slater's condition holds true, while for nonlinear systems a local reduction is possible, if a natural constraint qualification is satisfied. Finally, we show that the exact L p ‐penalization of state constraints with finite p is possible for convex problems, if Lagrange multipliers corresponding to the state constraints belong to L p′ , where p ′ is the conjugate exponent of p , and for general nonlinear problems, if the cost functional does not depend on the control inputs explicitly.

In this article we propose a new approach to an analysis of DC optimization problems. This approach was largely inspired by codifferential calculus and the method of codifferential descent and is based on the use of a so-called affine support set of a convex function instead of the Frenchel conjugate function. With the use of affine support sets we define a global codifferential mapping of a DC function and derive new necessary and sufficient global optimality conditions for DC optimization problems. We also provide new simple necessary and sufficient conditions for the global exactness of the ℓ _1 penalty function for DC optimization problems with equality and inequality constraints and present a series of simple examples demonstrating a constructive nature of the new global optimality conditions. These examples show that when the optimality conditions are not satisfied, they can be easily utilised in order to find “global descent” directions of both constrained and unconstrained problems. As an interesting theoretical example, we apply our approach to the analysis of a nonsmooth problem of Bolza.