We present a model of a commodity auction in which sellers and buyers (agents) represent risk using coherent risk measures. These are communicated to the auctioneer who computes socially optimal transactions assuming complete risk trading. The model is applied to economic dispatch and system marginal prices in a single-settlement wholesale electricity pool under uncertainty. If agents' risk measures are known by the system operator then prices form a socially optimal dispatch which is revenue adequate and recovers agents' costs in risk-adjusted expectation. We construct a non-cooperative game to show that agents have incentives to misrepresent their risk measures to improve their risk-adjusted profit.

We present a capacity expansion model for deciding the new electricity generation and transmission capacity to complement an existing hydroelectric reservoir system. The objective is to meet a forecast demand at least expected cost, namely the capital cost of the investment plus the expected discounted operating cost of the system. The optimal operating policy for any level of capacity investment can be computed using stochastic dual dynamic programming. We show how to combine a multistage stochastic operational model of the hydro system with a capacity expansion model to create a single model that can be solved by existing open-source solvers for multistage stochastic programs without the need for customized decomposition algorithms. We illustrate our method by applying it to a model of the New Zealand electricity system and comparing the solutions obtained with those found in a previous study.

We formulate and compare optimization models of investment in renewable generation using a suite of social planning models that compute optimal generation capacity investments for a hydro-dominated electricity system where inflow uncertainty results in a risk of energy shortage. The models optimize the expected cost of capacity expansion and operation allowing for investments in hydro, geothermal, solar, wind, and thermal plant, as well as battery storage for smoothing load profiles. A novel feature is the integration of uncertain seasonal hydroelectric energy supply and short-term variability in renewable supply in a two-stage stochastic programming framework. The models are applied to data from the New Zealand electricity system and used to estimate the costs of moving to a 100% renewable electricity system by 2035. We also explore the outcomes obtained when applying different forms of CO 2 constraint that limit respectively non-renewable capacity, non-renewable generation, and CO 2 emissions on average, almost surely, or in a chance-constrained setting, and show how our models can be used to investigate the merits of a proposed pumped-hydro scheme in New Zealand’s South Island.

Infinite programming is concerned with optimization problems in which the number of variables and the number of constraints are both possibly infinite. If either the constraints or variables are finite in number then an infinite program is called a semi-infinite program. In this paper we review some recent work which has been carried out in infinite programming, in particular in the fields of semi-infinite linear programming, and continuous-time linear programming. Some discussion is made of theoretical issues such as the existence of solutions and duality theory, but we put a greater emphasis on endeavouring to summarize and explain some of the algorithmic developments in these areas.

We apply the JuDGE optimization package to a multistage stochastic leader–follower model that determines a transmission capacity expansion plan to maximize expected social welfare of consumers and producers who act as Cournot oligopolists in each time period. The problem is formulated as a large-scale mixed integer programme and applied to a 5-bus instance over scenario trees of varying size. The computational effort required by JuDGE is compared with solving the deterministic equivalent mixed integer programme using a state-of-the-art integer programming package. This article is part of the theme issue ‘The mathematics of energy systems’.

The urgent need to decarbonize energy systems gives rise to many challenging areas of interdisciplinary research, bringing together mathematicians, physicists, engineers and economists. Renewable generation, especially wind and solar, is inherently highly variable and difficult to predict. The need to keep power and energy systems balanced on a second-by-second basis gives rise to problems of control and optimization, together with those of the management of liberalized energy markets. On the longer time scales of planning and investment, there are problems of physical and economic design. The papers in the present issue are written by some of the participants in a programme on the mathematics of energy systems which took place at the Isaac Newton Institute for Mathematical Sciences in Cambridge from January to May 2019-see http://www.newton.ac.uk/event/mes. This article is part of the theme issue 'The mathematics of energy systems'.

Sample average approximation is a popular approach to solving stochastic optimization problems. It has been widely observed that some form of robustification of these problems often improves the out-of-sample performance of the solution estimators. In estimation problems, this improvement boils down to a trade-off between the opposing effects of bias and shrinkage. This paper aims to characterize the features of more general optimization problems that exhibit this behaviour when a distributionally robust version of the sample average approximation problem is used. The paper restricts attention to quadratic problems for which sample average approximation solutions are unbiased and shows that expected out-of-sample performance can be calculated for small amounts of robustification and depends on the type of distributionally robust model used and properties of the underlying ground-truth probability distribution of random variables. The paper was written as part of a New Zealand funded research project that aimed to improve stochastic optimization methods in the electric power industry. The authors of the paper have worked together in this domain for the past 25 years.

We study a competitive partial equilibrium in markets where risk-averse agents solve multistage stochastic optimization problems formulated in scenario trees. The agents trade a commodity that is produced from an uncertain supply of resources. Both resources and the commodity can be stored for later consumption. Several examples of a multistage risked equilibrium are outlined, including aspects of battery and hydroelectric storage in electricity markets, distributed ownership of competing technologies relying on shared resources, and aspects of water control and pricing. The agents are assumed to have nested coherent risk measures based on one-step risk measures with polyhedral risk sets that have a nonempty intersection over agents. Agents can trade risk in a complete market of Arrow-Debreu securities. In this setting, we define a risk-trading competitive market equilibrium and establish two welfare theorems. Competitive equilibrium will yield a social optimum (with a suitably defined social risk measure) when agents have strictly monotone one-step risk measures. Conversely, a social optimum with an appropriately chosen risk measure will yield a risk-trading competitive market equilibrium when all agents have strictly monotone risk measures. The paper also demonstrates versions of these theorems when risk measures are not strictly monotone.

Mixed integer dynamic approximation scheme (MIDAS) is a new sampling-based algorithm for solving finite-horizon stochastic dynamic programs with monotonic Bellman functions. MIDAS approximates these value functions using step functions, leading to stage problems that are mixed integer programs. We provide a general description of MIDAS, and prove its almost-sure convergence to a $$2T\varepsilon $$-optimal policy for problems with T stages when the Bellman functions are known to be monotonic, and the sampling process satisfies standard assumptions.

Summary This paper addresses the challenge of design optimization under uncertainty when the designer only has limited data to characterize uncertain variables. We demonstrate that the error incurred when estimating a probability distribution from limited data affects the out‐of‐sample performance (ie, performance under the true distribution) of optimized designs. We demonstrate how this can be mitigated by reformulating the engineering design problem as a distributionally robust optimization (DRO) problem. We present computationally efficient algorithms for solving the resulting DRO problem. The performance of the DRO approach is explored in a practical setting by applying it to an acoustic horn design problem. The DRO approach is compared against traditional approaches to optimization under uncertainty, namely, sample‐average approximation and multiobjective optimization incorporating a risk reduction objective. In contrast with the multiobjective approach, the proposed DRO approach does not use an explicit risk reduction objective but rather specifies a so‐called ambiguity set of possible distributions and optimizes against the worst‐case distribution in this set. Our results show that the DRO designs, in some cases, significantly outperform those designs found using the sample‐average or the multiobjective approach.

Drilling geothermal wells has a very high capital cost. The location and operation of wells affects their production, so it is important to maximize value from wells by optimizing these decisions. The economic outcomes from particular well placement and operating policies can be estimated using reservoir simulations. A method has been developed to efficiently predict production outcomes from different combinations of possible wells and production starting times, using a relatively small number of reservoir simulations. This can then be used with optimization methods to select the best well locations and production starting times. A Mixed Integer Programming (MIP) model is presented to show this. Binary decision variables were used to select the combination of wells that would maximize total Net Present Value (NPV). Combined this approach provides an efficient method for finding optimal drilling plans, given a calibrated reservoir model.

We consider solving stochastic optimization problems in which we seek to minimize the expected value of an objective function with respect to an unknown distribution of random parameters. Our focus is on models that use sample average approximation (SAA) with small sample sizes. We analyse the out-of-sample performance of solutions obtained by solving a robust version of the SAA problem, and derive conditions under which these solutions are improved in comparison with SAA. We analyse three different mechanisms for constructing a robust solution: a CVaR-based risk measure, phi-divergence using total variation, and a Wasserstein metric.

Trading in the New Zealand wholesale electricity market (NZEM) pool began on 1 October 1996. The New Zealand pool market was the first one to use two elements of what has since become the North American standard market design, with locational marginal pricing and ancillary service cooptimization, but it was implemented in a physical and regulatory environment that differs markedly from that in North America. This article reflects on 20 years of experience with New Zealand's 1996 market design, assesses its points of difference from other jurisdictions, and speculates on some possible future directions.

Using a suite of perfectly competitive counterfactual models we study the effi ciency of the New Zealand wholesale electricity market over the ten-year period 2008-2017. Our models combine stochastic dual dynamic programming with a high fidelity simulation model of the New Zealand hydrothermal electricity system. Results of the models are given for the calendar years 2008-2017, and compared with wholesale market outcomes in those years. Counterfactual outcomes depend on assumptions on fuel prices and attitudes towards shortage risk. Depending on assumptions, we identify differences in productive effi ciency and Ricardian rents.

This paper provides a framework for deriving payment mechanisms for intermittent, flexible and inflexible electricity generators who are dispatched according to the optimal solution of a stochastic program that minimizes the expected cost of generation plus deviation. The first stage corresponds to a pre-commitment decision, and the second stage corresponds to real-time generation that adapts to different realizations of a random variable. By taking the Lagrangian and decoupling in different ways we study two payment mechanisms with different properties.

Pastoral dairy farmers make sequential decisions in the face of long-term environmental uncertainty and price volatility. Decisions made early in the season, such as the number of cows to stock per hectare, can have significant effects later in the season if the farmer is forced to import additional feed to meet the cows' energy demands during a drought. In this paper, we present POWDer: the milk Production Opti-mizer incorporating Weather Dynamics. POWDer is a novel multi-stage stochastic program that divides the dairy farming season into weeks and links these weeks by a system of linear dynamics. By applying POWDer to a case farm in New Zealand, we demonstrate POWDec's promise as a tool that can help participants in the New Zealand dairy industry understand and plan for the challenge of farming in a stochastic world. (C) 2018 Elsevier B.V. All rights reserved.

We discuss risked competitive partial equilibrium in a setting in which agents are endowed with coherent risk measures. In contrast to socialplanning models, we show by example that risked equilibria are not unique, even when agents' objective functions are strictly concave. We also show that standard computational methods find only a subset of the equilibria, even with multiple starting points.

Transmission constraints limit competition and arbitrageurs' possibilities of exploiting price differences between commodities in neighbouring markets. We analyze radial transmission-constrained networks with local demand shocks, where spatially distributed oligopoly producers compete with supply functions, as in wholesale electricity markets. We prove existence and uniqueness of supply-function equilibrium in two-node networks, and we are able to explicitly solve for symmetric supply-function equilibria in two-node and star networks. (C) 2018 Elsevier B.V. All rights reserved.

Deciding how to represent and manage uncertainty is a vital part of designing complex systems. Widely used is a probabilistic approach—assigning a probability distribution to each uncertain variable. However, this presents the designer with the task of assuming or estimating these probability distributions from data; a task which is inevitably prone to error. This paper addresses this challenge by formulating a distributionally robust design optimization problem, and presents computationally e � cient algorithms for solving the problem. In distributionally robust optimization (DRO) methods, the designer acknowl-edges that they are unable to exactly specify a probability distribution for the uncertain variables, and instead specifies a so-called ambiguity set of possible distributions. This paper uses an acoustic horn design problem to explore how the error incurred in estimating a probability distribution from data a ↵ ects the realized performance of designs found using a traditional multi-objective optimization under uncertainty. It is found that placing some importance on a risk reduction objective results in designs that are more robust to these errors, and thus have a better mean performance realized under the true distribution than if the designer were to focus all e ↵ orts on optimizing for mean performance alone. In contrast, the DRO approach is able to uncover designs that are not attainable using the multi-objective approach when given the same data. These DRO designs in some cases significantly outperform those designs found using the multi-objective approach.

Our model of strategic behavior in sequential markets exhibits a persistent forward price premium. This premium is not susceptible to arbitrage by speculators on the forward market, since purchasers prefer forward contracts backed by producers with market power.