Benchmark problems play a central role in assessing the performance of numerical optimization algorithms. However, many existing constrained multiobjective optimization benchmark problems rely on overly restricted constructions or lack formal analysis of their optimal solution sets, limiting their relevance for systematic algorithm evaluation. In this work, we introduce a class of analytically tractable constrained multiobjective optimization problems whose Pareto sets can be formally characterized. The construction is based on convex-quadratic functions with positive definite Hessians, combined through multipeak formulations in which each objective is defined as the minimum over several convex-quadratic components. This approach preserves analytical structure while enabling multimodality (non-convexity), ill-conditioning and non-separability. The constraints are built as sublevel sets of multipeak functions giving rise to problems with potentially disconnected feasible regions. Building on these results, we propose COBI, a scalable generator of constrained bi-objective test problems designed for benchmarking derivative-free optimization algorithms. We provide a reference Python implementation that enables straightforward integration of COBI instances into benchmarking workflows.

Mixed discrete–continuous optimization problems arise frequently in practice, yet remain algorithmically challenging. Although separable problems—where discrete and continuous components can be optimized independently—are often considered simple, we show that they can nonetheless present nontrivial challenges for algorithms. In this context, we introduce a refined notion of separability, termed ordinal separability, which generalizes additive decomposability and encompasses, for instance, products of positive functions, while avoiding limitations of a previous definition. We show that, for ordinally separable functions and comparison-based subspace optimizers, alternating optimization generates the same trajectory as independently optimizing each subspace subsequently. We demonstrate, on the contrary, that even the simplest separable mixed problems pose difficulties for integrated optimization approaches that apply simultaneous discrete and continuous variation operators. Simple benchmarking experiments show that combining bit-flip mutations with Gaussian mutations and success-based step size adaptation can suffer from premature convergence. This failure arises from indirect coupling between discrete and continuous components through selection, highlighting fundamental issues in mixed discrete–continuous problems, even in the most favorable separable setting.

We introduce and analyze the (1+1)-EA-ES for mixed discrete–continuous optimization, combining a bitflip operator for discrete coordinates with a Gaussian mutation and a step size adapted via success rules for continuous coordinates. We show that the discrete component of the search—especially when close to optimal—can hinder progress in the continuous part by reducing the overall success probability. This disrupts the step size adaptation mechanism of the continuous component and may lead to premature convergence. We formally prove that on SphereOneMax, for large enough dimensions, a reduced target success probability can bypass this problem. More precisely, we prove for the (1+1)-EA-ES with reduced success probability a bound on the expected time to reach an epsilon neighborhood of the optimum in 𝒪( max{ n_c,n_d }·( log (1/ε ) + n_d + 1/n_c ) ) where n_c represents the number of continuous variables and n_d the number of discrete ones.

We introduce linear surrogate functions for modeling inequality constraints to solve constrained blackbox optimization problems with the Augmented Lagrangian CMA-ES. Each surrogate is constructed from a binary classifier that predicts the sign of the constraint value. The classifier, and consequently the resulting algorithm, is invariant under sign preserving transformations of the constraint values and can handle binary, flat, and deceptive constraints. Somewhat surprisingly, we find that adopting a sign-based classification model of the constraints allows to solve classes of constrained problems which can not be solved with the original Augmented Lagrangian method using the true constraint value.

In this poster, we introduce a rank-based surrogate-assisted variant of CMA-ES. Unlike previous methods that employ rank information as constraints to train an SVM classifier, our approach employs a linear-quadratic regression on the ranks. We investigate the method's invariance empirically. While this first algorithm outperforms CMA-ES with a few exceptions, it falls short to entirely meet the lq-CMA-ES performance levels. To address this, we propose an enhanced variant that handles together two alternative surrogates, one based on the ranks and one based on the original function values. Although this variant sacrifices strict invariance, it gains in robustness and achieves performance comparable to, or even exceeding, lq-CMA-ES on transformed problems. This last algorithm shows how simply incorporating new transformations of rank values could improve any surrogate-based CMA-ES variant.

We analyze a fundamental class of multiobjective constrained problems where the objectives are spherical functions and the constraints are convex. As an application from the projection theorem on closed convex sets, we prove that the constrained Pareto set corresponds to the orthogonal projection of the unconstrained Pareto set onto the feasible region. We establish this fundamental geometric property and illustrate its implications using visualizations of Pareto sets and fronts under various constraint configurations. Furthermore, we assess the performance of NSGA-II on these problems, examining its ability to approximate the constrained Pareto set across different dimensions. Our findings highlight the importance of theoretically grounded and understood benchmark problems for assessing algorithmic behavior and contribute to a deeper understanding of constrained multiobjective landscapes.

The pdfo library by Tom M. Ragonneau and Zaikun Zhang makes the five derivative-free solvers BOBYQA, COBYLA, LINCOA, NEW-UOA, and UOBYQA-originally written by Michael J. D. Powell-available in Python. In this paper, we are comparing their performance on the bbob test suite with three other solvers from the COCO data archive: CMA-ES from pycma, SLSQP and BFGS from scipy. We also compare the original solvers, written by Powell in Fortran 77, with the current pdfo versions, which saw multiple bug fixes and code improvements by Ragonneau and Zhang. For the latter comparison, we do not see large effects on performance between the Fortran 77 version and the current pdfo version. The only notable exception is the Bent Cigar function where we observe differences by a factor of 25 for BOBYQA, LINCOA, and NEWUOA. Compared to the other baseline algorithms, BOBYQA, LINCOA and NEWUOA perform very similarly over all bbob functions, being about a factor of 5 slower than SLSQP and BFGS while UOBYQA-as the best-performing pdfo solver-outperforms SLSQP and BFGS for larger budgets when compared over all 24 bbob functions. The linear surrogate of COBYLA, on the contrary, is clearly worse over all functions than the other algorithms.

UOBYQA, short for Unconstrained Optimization By Quadratic Approximation, is one of the well-known solvers derived and implemented by Michael J. D. Powell. In each step, the algorithm builds a quadratic surrogate of the objective function, interpolating quadratically many points for which the true function values are known. The model is optimized within the so-called trust region and the resulting solution is evaluated next. Adaptation of the trust region radius allows for fast convergence on a wide range of (noiseless) functions without the need for derivatives. In this workshop paper, we investigate the effect of (frozen) nonnegative, i.e., worsening noise on UOBYQA with varying probability of solutions being affected by the noise. To this end, we use the COCO platform and its newest addition, the noiser, applied to the classical bbob functions. The numerical benchmarking experiments showcase that UOBYQA is negatively affected by the noise, but surprisingly little over a wide range of noise strengths for some of the bbob functions.

The Borg MOEA [8] is an optimization algorithm, designed to handle real-world problems of a multi objective and multimodal nature. In this report, we examine the effectiveness of Borg for solving optimization problems with only two objectives. To this end, we benchmark the performance of the algorithm on the bbob-biobj test suite via the COCO platform, comparing it to current state-of-the-art algorithms. The study uses standard values for all the parameters but one, as retrieved from http://borgmoea.org/. The only parameter that varies between different problem instances is the ε parameter, a crucial scale tuning parameter. To adapt this parameter, we devised and applied a heuristic. We find that the algorithm performs respectably, although it does not surpass the current state-of-the-art algorithms for any of the problem instances examined, and particularly loses performance on problems with a high-dimensional search space. Additionally, we observed that our heuristic for tuning the ε-parameter results in significant performance improvements compared to using a fixed value for ε.

We present a surrogate-assisted multiobjective optimization algorithm. The aggregation of the objectives relies on the Uncrowded Hypervolume Improvement (UHVI) which is partly replaced by a linear-quadratic surrogate that is integrated into the CMA-ES algorithm. Surrogating the UHVI poses two challenges. First, the UHVI is a dynamic function, changing with the empirical Pareto set. Second, it is a composite function, defined differently for dominated and nondominated points. The presented algorithm is thought to be used with expensive functions of moderate dimension (up to about 50) with a quadratic surrogate which is updated based on its ranking ability. We report numerical experiments which include tests on the COCO benchmark. The algorithm shows in particular linear convergence on the double sphere function with a convergence rate that is 6--20 times faster than without surrogate assistance.

Bound constraints on the variables are the most basic constraints in an optimization problem formulation and, thus, among the most common. It is therefore essential to understand the impact of different boundary handling techniques on algorithm performance. Equally, it is important to understand the practical impact of using bound constraint handling in an algorithm on principally unbounded problems but where the user has a good indication of the domain of the (sought) optimum. Both questions will be investigated in this paper on the newly introduced box-constrained version sbox-cost of the well-known, unconstrained test suite bbob and for the example of the two boundary handling techniques, implemented in the CMA-ES python module pycma. The numerical experiments performed with the COCO platform show that there is (i) only a minor difference in performance between the two test suites and (ii) a slight performance reduction for the (default) BoundTransform boundary handling compared to the BoundPenalty version of CMA-ES.

We present concepts and recipes for the anytime performance assessment when benchmarking optimization algorithms in a blackbox scenario. We consider runtime—oftentimes measured in the number of blackbox evaluations needed to reach a target quality—to be a universally measurable cost for solving a problem. Starting from the graph that depicts the solution quality versus runtime, we argue that runtime is the only performance measure with a generic, meaningful, and quantitative interpretation. Hence, our assessment is solely based on runtime measurements. We discuss proper choices for solution quality indicators in single- and multi-objective optimization, as well as in the presence of noise and constraints. We also discuss the choice of the target values, budget-based targets, and the aggregation of runtimes by using simulated restarts, averages, and empirical cumulative distributions which generalize convergence graphs of single runs. The presented performance assessment is to a large extent implemented in the comparing continuous optimizers (COCO) platform freely available at https://github.com/numbbo/coco .

tutorial Share on GECCO 2022 tutorial on benchmarking multiobjective optimizers 2.0 Authors: Dimo Brockhoff Inria InriaView Profile , Tea Tušar Jožef Stefan Institute Jožef Stefan InstituteView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 1269–1309https://doi.org/10.1145/3520304.3533635Published:19 July 2022Publication History 0citation32DownloadsMetricsTotal Citations0Total Downloads32Last 12 Months32Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Several test function suites are being used for numerical benchmarking of multiobjective optimization algorithms. While they have some desirable properties, such as well-understood Pareto sets and Pareto fronts of various shapes, most of the currently used functions possess characteristics that are arguably underrepresented in real-world problems such as separability, optima located exactly at the boundary constraints, and the existence of variables that solely control the distance between a solution and the Pareto front. Via the alternative construction of combining existing single-objective problems from the literature, we describe the bbob-biobj test suite with 55 bi-objective functions in continuous domain, and its extended version with 92 bi-objective functions (bbob-biobj-ext). Both test suites have been implemented in the COCO platform for black-box optimization benchmarking and various visualizations of the test functions are shown to reveal their properties. Besides providing details on the construction of these problems and presenting their (known) properties, this article also aims at giving the rationale behind our approach in terms of groups of functions with similar properties, objective space normalization, and problem instances. The latter allows us to easily compare the performance of deterministic and stochastic solvers, which is an often overlooked issue in benchmarking.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). GECCO '14, Jul 12-16 2014, Vancouver, BC, Canada ACM 978-1-4503-2881-4/14/07. http://dx.doi.org/10.1145/2598394.2605339 Tea Tušar tea.tusar@ijs.si

Direct Multisearch (DMS) and MultiGLODS are two derivative-free solvers for approximating the entire set of Pareto-optimal solutions of a multiobjective (blackbox) problem. They both follow the search/poll step approach of direct search methods, employ Pareto dominance to avoid aggregating objectives, and have theoretical limit guarantees. Although the original publications already compare the two algorithms empirically with a variety of multiobjective solvers, an analysis on their scaling behavior with dimension was missing. Here, we run the publicly available implementations on the bbob-biobj test suite of the COCO platform and by investigating their performances in more detail, observe (i) a small defect in the default initialization of DMS, (ii) for both algorithms a decrease in relative performance to other algorithms of the original studies (even matching the performance of random search for MultiGLODS in higher dimension), and (iii) consequently, an under-performance to previously untested stochastic solvers from the evolutionary computation field, especially when the dimension is higher.