Large amounts of energy are consumed and greenhouse gasses are emitted in the process of extracting oil and gas from offshore fields. We investigate methods for solving the multi-objective production optimization for minimizing the emissions of CO2 while maximizing economic profitability. Two scalarization methods for solving the multi-objective optimization problem - the weighted-sum method and the epsilon-constraint method - are evaluated for computational efficiency and interpretability of solutions on a simplified model of a North Sea oil field. The epsilon-constraint approach provided an easily interpretable optimization problem with the ability to specify a target reduction in CO2 emissions, but was relatively expensive to apply. The weighted-sum approach was easier to implement and the cost to obtain a solution was less than the cost for the epsilon-constraint approach by a factor of approximately three. For the injection rate controls in our example application, there were gaps in the Pareto optimal solutions obtained using the weighted sum approach when large weights were applied to the emissions reduction objective function.

Ensemble-based optimization (EnOpt), commonly used in reservoir management, can be seen as a special case of a natural evolution algorithm. Stein’s lemma gives a new interpretation of EnOpt. This interpretation enables us to study EnOpt in the context of general mutation distributions. In this paper, a non-Gaussian generalization of EnOpt (GenOpt) is proposed, where the control gradient is estimated using Stein’s lemma, and the mutation distribution is updated separately via natural evolution. For the multivariate case, a Gaussian copula is used to represent dependencies between the marginals. The correlation matrix is also iteratively optimized. It is shown that using beta distributions as marginals in the GenOpt algorithm addresses the truncation problem that sometimes arises when applying EnOpt on bounded optimization problems. The performance of the proposed optimization algorithm is evaluated on several test cases. The experiments indicate that GenOpt is less dependent on the chosen hyperparameters, and it is able to converge more quickly than EnOpt on a reservoir management test case.

We investigate the use of various momentum methods in combination with an ensemble approximation of gradients, for accelerated optimization. Although momentum gradient descent methods are popular in machine learning, it is unclear how they perform when applied to time-consuming dynamic problems such as production optimization for petroleum reservoir management. Four different momentum methods are extensively tested on a reservoir test case in one deterministic and one robust setting. The numerical experiments show that momentum strategies yield, on average, a higher net present value with fewer simulations needed.

Data assimilation (DA) is the science of optimally combining sparse data and computational models, both of which are typically large and varied.Based on classical statistics, signal processing, and control systems theory, and increasingly, machine learning, DA was primarily developed in weather, climate, and oceanographic forecasting but is presently used all across the geosciences and beyond.DAPPER is a set of templates for benchmarking the performance of DA methods.Figure 1: Screenshot from "liveplotting" in DAPPER

In robust optimisation problems the objective function consists of an average over (an ensemble of) uncertain parameters. Ensemble optimisation (EnOpt) implements steepest descent by estimating the gradient using linear regression on Monte-Carlo simulations of (an ensemble of) control parameters. Applying EnOpt for robust optimisation is costly unless the evaluations over the two ensembles are combined, i.e. 'paired'. Here, we provide a new and more rigorous perspective on the stochastic simplex approximate gradient (StoSAG) used in EnOpt, explaining how it addresses detrimental cross-correlations arising from pairing by only capturing the variability due to the control vector, and not the vector of uncertain parameters. A few minor variants are derived from a generalised derivation, as well as a new approach using decorrelation. These variants are tested on linear and non-linear toy gradient estimation problems, where they achieve highly similar accuracy, but require a very large ensemble size to outperform the non-robust approach when accounting for variance and not just bias. Other original contributions include a discussion of the particular robust control objectives for which EnOpt is suited, illustrations, a variance reduction perspective, and a discussion on the centring in covariance and gradient estimation.

Chaos is ubiquitous in physical systems. The associated sensitivity to initial conditions is a significant obstacle in forecasting the weather and other geophysical fluid flows. Data assimilation is the process whereby the uncertainty in initial conditions is reduced by the astute combination of model predictions and real-time data. This chapter reviews recent findings from investigations on the impact of chaos on data assimilation methods: for the Kalman filter and smoother in linear systems, analytic results are derived; for their ensemble-based versions and nonlinear dynamics, numerical results provide insights. The focus is on characterising the asymptotic statistics of the Bayesian posterior in terms of the dynamical instabilities, differentiating between deterministic and stochastic dynamics. We also present two novel results. Firstly, we study the functioning of the ensemble Kalman filter in the context of a chaotic, coupled, atmosphere-ocean model with a quasi-degenerate spectrum of Lyapunov exponents, showing the importance of having sufficient ensemble members to track all of the near-null modes. Secondly, for the fully non-Gaussian method of the particle filter, numerical experiments are conducted to test whether the curse of dimensionality can be mitigated by discarding observations in the directions of little dynamical growth of uncertainty. The results refute this option, most likely because the particles already embody this information on the chaotic system. The results also suggest that it is the rank of the unstable-neutral subspace of the dynamics, and not that of the observation operator, that determines the required number of particles. We finally discuss how knowledge of the random attractor can play a role in the development of future data assimilation schemes for chaotic multiscale systems with large scale separation.

CrowdEmotion produce software to measure a person's emotions based on analysis of microfacial expressions using a machine learning algorithm to recognize which features correspond with which emotions. The features are derived by applying a bank of Gabor filters to a set of frames. CrowdEmotion needed to improve the accuracy, processing speed and cost-efficiency of the tool. In particular they wanted to know if a subset of the bank of Gabor filters was sufficient, and whether the image filtering stage could be implemented on a GPU. A framework for choosing the optimum set of Gabor filters was established and ways of reducing the dimensionality of this were interrogated. Taking a subset of Local Binary Patterns was found to be fully justified. Meanwhile choosing a gridding pattern is open to interpretation; some suggestions were made about how this choice might be improved.

A Bayesian method of inference known as “Stein variational gradient descent” was recently implemented for data assimilation problems, under the heading of “mapping particle filter”. In this manuscript, the algorithm is applied to another type of geoscientific inversion problems, namely history matching of petroleum reservoirs. In order to combat the curse of dimensionality, the commonly used Gaussian kernel, which defines the solution space, is replaced by a p-kernel. In addition, the ensemble gradient approximation used in the mapping particle filter is rectified, and the data assimilation experiments are re-run with more relevant settings and comparisons. Our experimental results in data assimilation are rather disappointing. However, the results from the subsurface inverse problem show more promise, especially as regards the use of p-kernels.

Summary We have introduced and applied the Stein variational gradient descent (SVGD) algorithm to the reservoir history matching problem. The method has been extended with a Monte Carlo approximation of the gradient of the measurement function using the reproducible property of the reproducing kernel Hilbert space in order to avoid adjoint implementation. Furthermore we have implemented SVGD with p-kernels in order to extend the applicability to higher dimension than the standard Gaussian kernels used in the literature. The gradient approximation was validated both theoretically and on a toy model. Implementation on larger problems will be addressed in the future. We also showed using a reservoir model that the SVGD with p-kernels provided better estimates of the uncertainty than the standard Gaussian kernels. Further comparison with ensemble-based methods and other gradient-based methods is left for future research.

Raanes et al. [1] revised the iterative ensemble smoother of Chen and Oliver [2, 3], denoted Ensemble Randomized Maximum Likelihood (EnRML), using the property that the EnRML solution is contained in the ensemble subspace. They analyzed EnRML and demonstrated how to implement the method without the use of expensive and potentially unstable pseudo inversions of the low-rank state covariance matrix or the ensemble-anomaly matrix. The new algorithm produces the same result, realization by realization, as the original EnRML method. However, the new formulation is simpler to implement, numerically stable, and computationally more efficient. The purpose of this document is to present a simple derivation of the new algorithm and demonstrate its practical implementation and use for reservoir history matching. An additional focus is to customize the algorithm to be suitable for big-data assimilation of measurements with correlated errors. We demonstrate that the computational cost of the resulting “ensemble sub-space” algorithm is linear in the number of measurements, also when the measurements have correlated errors, as well as the state-space dimension. The final algorithm is implemented in the Ensemble Reservoir Tool (ERT) for running and conditioning ensembles of reservoir models. Several verification experiments are presented.

Ensemble randomized maximum likelihood (EnRML) is an iterative (stochastic) ensemble smoother, used for large and nonlinear inverse problems, such as history matching and data assimilation. Its current formulation is overly complicated and has issues with computational costs, noise, and covariance localization, even causing some practitioners to omit crucial prior information. This paper resolves these difficulties and streamlines the algorithm without changing its output. These simplifications are achieved through the careful treatment of the linearizations and subspaces. For example, it is shown (a) how ensemble linearizations relate to average sensitivity and (b) that the ensemble does not lose rank during updates. The paper also draws significantly on the theory of the (deterministic) iterative ensemble Kalman smoother (IEnKS). Comparative benchmarks are obtained with the Lorenz 96 model with these two smoothers and the ensemble smoother using multiple data assimilation (ES-MDA).

This paper studies multiplicative inflation: the complementary scaling of the state covariance in the ensemble Kalman filter (EnKF). Firstly, error sources in the EnKF are catalogued and discussed in relation to inflation; nonlinearity is given particular attention as a source of sampling error. In response, the "finite-size" refinement known as the EnKF-N is re-derived via a Gaussian scale mixture, again demonstrating how it yields adaptive inflation. Existing methods for adaptive inflation estimation are reviewed, and several insights are gained from a comparative analysis. One such adaptive inflation method is selected to complement the EnKF-N to make a hybrid that is suitable for contexts where model error is present and imperfectly parameterized. Benchmarks are obtained from experiments with the two-scale Lorenz model and its slow-scale truncation. The proposed hybrid EnKF-N method of adaptive inflation is found to yield systematic accuracy improvements in comparison with the existing methods, albeit to a moderate degree.

The Rauch–Tung–Striebel (RTS) smoother is a linear‐Gaussian smoothing algorithm that is popular in the engineering community. This note is a study of its ensemble formulation (EnRTS). An on‐line expression is derived and discussed. In particular, it is used to show that the EnRTS is equivalent to the ensemble Kalman smoother (EnKS), even in the nonlinear, non‐Gaussian case. The theory is revisited under practical considerations and equability is illustrated by numerical experiments, even though equivalence is broken by inflation and localisation.

The ensemble Kalman filter (EnKF) is a powerful data assimilation method meant for high-dimensional nonlinear systems. But its implementation requires somewhat ad hoc procedures such as localization and inflation. The recently developed finite-size ensemble Kalman filter (EnKF-N) does not require multiplicative inflation meant to counteract sampling errors. Aside from the practical interest in avoiding the tuning of inflation in perfect model data assimilation experiments, it also offers theoretical insights and a unique perspective on the EnKF. Here, we revisit, clarify and correct several key points of the EnKF-N derivation. This simplifies the use of the method, and expands its validity. The EnKF is shown to not only rely on the observations and the forecast ensemble, but also on an implicit prior assumption, termed hyperprior, that fills in the gap of missing information. In the EnKF-N framework, this assumption is made explicit through a Bayesian hierarchy. This hyperprior has so far been chosen to be the uninformative Jeffreys prior. Here, this choice is revisited to improve the performance of the EnKF-N in the regime where the analysis is strongly dominated by the prior. Moreover, it is shown that the EnKF-N can be extended with a normal-inverse Wishart informative hyperprior that introduces additional information on error statistics. This can be identified as a hybrid EnKF–3D-Var counterpart to the EnKF-N.

AbstractA square root approach is considered for the problem of accounting for model noise in the forecast step of the ensemble Kalman filter (EnKF) and related algorithms. The primary aim is to replace the method of simulated, pseudo-random additive so as to eliminate the associated sampling errors. The core method is based on the analysis step of ensemble square root filters, and consists in the deterministic computation of a transform matrix. The theoretical advantages regarding dynamical consistency are surveyed, applying equally well to the square root method in the analysis step. A fundamental problem due to the limited size of the ensemble subspace is discussed, and novel solutions that complement the core method are suggested and studied. Benchmarks from twin experiments with simple, low-order dynamics indicate improved performance over standard approaches such as additive, simulated noise, and multiplicative inflation.

Data assimilation considers the problem of using a variety of data to calibrate model-based estimates of dynamic variables and static parameters. Geoscientific examples include (i) satellite observations and atmospheric models for weather forecasting, and (ii) well-log data and reservoir flow simulators for oil production optimization. Approximate solutions are provided by the set of techniques deriving from the ensemble Kalman filter (EnKF), which combines a Monte Carlo approach with assumptions of linearity and Gaussianity. This thesis proposes some improvements to the accuracy and understanding of such ensemble methods. Firstly, a new scheme is developed to account for model noise in the forecast step of the EnKF. The main aim is to eliminate the sampling errors of additive, simulated noise. The scheme is based on the previously developed schemes for the analysis step, but requires further consideration due to the limited subspace spanned by the ensemble. The properties of the square root scheme in general are surveyed. Secondly, the finite size ensemble Kalman filter (EnKF-N) is reviewed. The EnKF-N explicitly considers the uncertainty in the forecast moments (mean and covariance), thereby not requiring the multiplicative inflation commonly used to compensate for an intrinsic bias of the analysis step of the standard EnKF. Thus, in the perfect model setting, it avoids the process of tuning the inflation factor. This presentation consolidates the earlier literature on the EnKF-N, substantiates the scalar inflation perspective, and rectifies a deficiency. Thirdly, two ensemble smoothers expressed by different recursions, used in different applications, and hitherto thought to yield different results, are shown to be equivalent. The theory is revisited under practical considerations, where equivalence is broken due to inflation and localization, but the methods remain equally capable. In each case, the theory is tested and the accuracy performance is benchmarked against standard methods using numerical twin experiments.

We do a comparative study of the spacial con gurations of particles that arise from the stochastic processes of Random Sequential Adsorption (RSA) and Brownian Dynamics (BD). We relate this to biochemical application areas and, in particular, crowded environments. We describe some of the necessary theory of BD, RSA and kernel density estimation, and we develop a variety of BD simulation algorithms to tackle issues relating to collision handling. We undertake studies in one and two dimensions, and show that classical RSA does not generate the same type of particle con gurations as BD, although the di erence between the two is largely dependent on factors such as crowdedness and polydisperisty. In one dimension, we achieve better approximations to BD particle con gurations through simple modi cations of the RSA procedure.