High-dimensional problems have long been considered the Achilles' heel of Bayesian optimization algorithms. Spurred by the curse of dimensionality, a large collection of algorithms aim to make it more performant in this setting, commonly by imposing various simplifying assumptions on the objective. In this paper, we identify the degeneracies that make vanilla Bayesian optimization poorly suited to high-dimensional tasks, and further show how existing algorithms address these degeneracies through the lens of lowering the model complexity. Moreover, we propose an enhancement to the prior assumptions that are typical to vanilla Bayesian optimization algorithms, which reduces the complexity to manageable levels without imposing structural restrictions on the objective. Our modification - a simple scaling of the Gaussian process lengthscale prior with the dimensionality - reveals that standard Bayesian optimization works drastically better than previously thought in high dimensions, clearly outperforming existing state-of-the-art algorithms on multiple commonly considered real-world high-dimensional tasks.

Bayesian optimization is an effective method for optimizing expensive-to-evaluate black-box functions. High-dimensional problems are particularly challenging as the surrogate model of the objective suffers from the curse of dimensionality, which makes accurate modeling difficult. We propose a group testing approach to identify active variables to facilitate efficient optimization in these domains. The proposed algorithm, Group Testing Bayesian Optimization (GTBO), first runs a testing phase where groups of variables are systematically selected and tested on whether they influence the objective. To that end, we extend the well-established theory of group testing to functions of continuous ranges. In the second phase, GTBO guides optimization by placing more importance on the active dimensions. By exploiting the axis-aligned subspace assumption, GTBO is competitive against state-of-the-art methods on several synthetic and real-world high-dimensional optimization tasks. Furthermore, GTBO aids in the discovery of active parameters in applications, thereby enhancing practitioners' understanding of the problem at hand.

Robotic systems for manipulation tasks are increasingly expected to be easy to configure for new tasks. While in the past, robot programs were often written statically and tuned manually, the current, faster transition times call for robust, modular and interpretable solutions that also allow a robotic system to learn how to perform a task. We propose the method Behavior-based Bayesian Optimization and Planning (BeBOP) that combines two approaches for generating behavior trees: we build the structure using a reactive planner and learn specific parameters with Bayesian optimization. The method is evaluated on a set of robotic manipulation benchmarks and is shown to outperform state-of-the-art reinforcement learning algorithms by being up to 46 times faster while simultaneously being less dependent on reward shaping. We also propose a modification to the uncertainty estimate for the random forest surrogate models that drastically improves the results.

We introduce the Bayesian Compiler Optimization framework (BaCO), a general purpose autotuner for modern compilers targeting CPUs, GPUs, and FPGAs. BaCO provides the flexibility needed to handle the requirements of modern autotuning tasks. Particularly, it deals with permutation, ordered, and continuous parameter types along with both known and unknown parameter constraints. To reason about these parameter types and efficiently deliver high-quality code, BaCO uses Bayesian optimiza tion algorithms specialized towards the autotuning domain. We demonstrate BaCO's effectiveness on three modern compiler systems: TACO, RISE ELEVATE, and HPVM2FPGA for CPUs, GPUs, and FPGAs respectively. For these domains, BaCO outperforms current state-of-the-art autotuners by delivering on average 1.36x-1.56x faster code with a tiny search budget, and BaCO is able to reach expert-level performance 2.9x-3.9x faster.

Gaussian processes are the model of choice in Bayesian optimization and active learning. Yet, they are highly dependent on cleverly chosen hyperparameters to reach their full potential, and little effort is devoted to finding good hyperparameters in the literature. We demonstrate the impact of selecting good hyperparameters for GPs and present two acquisition functions that explicitly prioritize hyperparameter learning. Statistical distance-based Active Learning (SAL) considers the average disagreement between samples from the posterior, as measured by a statistical distance. SAL outperforms the state-of-the-art in Bayesian active learning on several test functions. We then introduce Self-Correcting Bayesian Optimization (SCoreBO), which extends SAL to perform Bayesian optimization and active learning simultaneously. SCoreBO learns the model hyperparameters at improved rates compared to vanilla BO, while outperforming the latest Bayesian optimization methods on traditional benchmarks. Moreover, we demonstrate the importance of self-correction on atypical Bayesian optimization tasks.

In liner shipping, containers are generally transshipped in major hub ports in each region, between larger inter-region vessels and smaller feeder vessels. In this paper, we study the problem of designing the feeder vessel network, transporting containers between a regional hub and the surrounding feeder ports. The problem, as modelled, has many similarities with the split delivery vehicle routing problem, but with additional characteristics such as simultaneous pickups and deliveries and weekly departures. The problem also includes fleet sizing with a heterogeneous fleet and allows demand rejection at a penalty cost. We present a branch-and-price framework to solve the problem, where the subproblem is solved by enumerating vessel routes and subsequently assigning commodities by solving a min-cost flow problem for each commodity. The algorithm is tested on instances with up to 12 ports, which are all solved to optimality. Since the problem is similar to the liner shipping network design problem but without transshipments, we further study selected instances from the LINER-LIB instance suite. The Baltic instance is solved to proven optimality and we find the best known solution to the West Africa instance, significantly improving on what can be found in the literature. (c) 2021 Published by Elsevier B.V.

This paper introduces the transit time constrained fixed charge multi-commodity network design problem. Transit times are origin-to-destination time limits for the commodities, which appear for example in transport systems with perishable goods. We discuss how to model the problem and present three different formulations of it. The first formulation is an exponential size path formulation, which we solve with a branch-and-price algorithm. Several speed up techniques from the literature on fixed charge multi-commodity network design problems are implemented, such as lifted cover inequalities and the recently proposed deep dual-optimal inequalities. In an extensive set of computational experiments, we show that these inequalities significantly improve the performance of the algorithm. The other two formulations are of polynomial size: one uses path indices and the other uses time indices. While the branch-and-price algorithm outperforms solving the compact formulations with a general-purpose mixed-integer programming solver, the study of compact models helps better understand the problem, and we can use them as benchmarks. A detailed sensitivity analysis of the branch-and-price algorithm shows that longer transit times and an increased ratio of fixed charge to flow cost increase the difficulty of solving the problem whereas the arc capacity has less impact. We further discuss in-depth implementational details.

Capacitated fixed-charge network design problems and generalizations, such as service network design problems, have a wide range of applications but are known to be very difficult to solve. Many exact and heuristic algorithms to solve these problems rely on column-and-row generation (CRG), which frequently suffer from primal degeneracy. We present a set of dual inequalities, equivalent to a simple primal relaxation, that speed up CRG algorithms for generalized capacitated fixed charge network design problems. We investigate the impact of the dual inequalities theoretically as well as experimentally. For practical applications, the presented technique is simple to implement, has no additional computational cost and can accelerate CRG by orders of magnitude, depending on the problem size and structure.

This paper proposes an Adaptive Large Neighbourhood Search heuristic for solving the Port Scheduling Problem, the problem of scheduling feeder vessels' operations in multi-terminal ports. Each vessel has a number of operations to perform at different terminals within the port, and each terminal can only serve a single vessel at a time. The resulting problem is a general shop-like problem, with a variety of additional operational constraints. The objective is to let the vessels depart from the port as early as possible, as this allows them to sail at reduced speed to the next port, saving large amounts of fuel; as well as scheduling operations early, which leaves more slack for later and hence makes the system more robust. The developed Adaptive Large Neighbourhood Search heuristic works with the order of operations, and assigns the start times of the operations first as part of the solution evaluation. To conduct the computational experiments, a large set of benchmark test instances, denoted PortLib, was developed, and the performance of the heuristic was compared to that of a commercial solver. The results show that the heuristic in general finds better solutions, even with significantly shorter run times. (C) 2020 Elsevier B.V. All rights reserved.

The maritime industry is one of the greenest modes of transportation, taking care of almost 90% of the global trade. The maritime container business revolves around liner shipping, which consists of container vessels sailing on fixed itineraries. For the last 20 years, there has been an increasing number of publications regarding how to design such fixed routes (services), to ensure a high level of service while minimizing operational costs and environmental impact. The liner shipping network design problem can briefly be described as follows: Given a set of demands (defined by origin, destination, time limit) and a set of vessels with variable capacity, the task is to design a set of weekly services, assign vessels to the services, and flow the demand through the resulting network such that it arrives within the stated time constraints. The objective is to maximize revenue of transported demand subtracting the operational costs. We present an in-depth literature overview of existing models and solution methods for liner shipping network design, and discuss the four main families of solution methods: integrated mixed integer programming models; two-stage algorithms designing services in the first step and flowing containers in the second step; two-stage algorithms first flowing containers and then designing services; and finally algorithms for selecting a subset of proposed candidate services. We end the presentation by comparing the performance of leading algorithms using the public LINER-LIB instances. The paper is concluded by discussing future trends in liner shipping, indicating directions for future research. (C) 2019 Elsevier B.V. All rights reserved.