We consider the problem of sampling from a probability distribution π. It is well known that this can be written as an optimisation problem over the space of probability distribution in which we aim to minimise the Kullback–Leibler divergence from π. We consider several partial differential equations (PDEs) whose solution is a minimiser of the Kullback–Leibler divergence from π and connect them to well-known Monte Carlo algorithms. We focus in particular on PDEs obtained by considering the Wasserstein–Fisher–Rao geometry over the space of probabilities and show that these lead to a natural implementation using importance sampling and sequential Monte Carlo. We propose a novel algorithm to approximate the Wasserstein–Fisher–Rao flow of the Kullback–Leibler divergence and conduct an extensive empirical study to identify when these algorithms outperforms other popular Monte Carlo algorithms.

In the domain of medical imaging, many supervised learning based methods for segmentation face several challenges such as high variability in annotations from multiple experts, paucity of labelled data and class imbalanced datasets. These issues may result in segmentations that lack the requisite precision for clinical analysis and can be misleadingly overconfident without associated uncertainty quantification. This work proposes the PULASki method as a computationally efficient generative tool for biomedical image segmentation that accurately captures variability in expert annotations, even in small datasets. This approach makes use of an improved loss function based on statistical distances in a conditional variational autoencoder structure (Probabilistic UNet) , which improves learning of the conditional decoder compared to the standard cross-entropy particularly in class imbalanced problems. The proposed method was analysed for two structurally different segmentation tasks (intracranial vessel and multiple sclerosis (MS) lesion) and compare our results to four well-established baselines in terms of quantitative metrics and qualitative output. These experiments involve class-imbalanced datasets characterised by challenging features, including suboptimal signal-to-noise ratios and high ambiguity. Empirical results demonstrate the PULASKi method outperforms all baselines at the 5% significance level. Our experiments are also of the first to present a comparative study of the computationally feasible segmentation of complex geometries using 3D patches and the traditional use of 2D slices. The generated segmentations are shown to be much more anatomically plausible than in the 2D case, particularly for the vessel task. Our method can also be applied to a wide range of multi-label segmentation tasks and is useful for downstream tasks such as hemodynamic modelling (computational fluid dynamics and data assimilation), clinical decision making, and treatment planning.

The aim of this paper is to obtain convergence in mean in the uniform topology of piecewise linear approximations of Stochastic Differential Equations (SDEs) with $C^1$ drift and $C^2$ diffusion coefficients with uniformly bounded derivatives. Convergence analyses for such Wong-Zakai approximations most often assume that the coefficients of the SDE are uniformly bounded. Almost sure convergence in the unbounded case can be obtained using now standard rough path techniques, although $L^q$ convergence appears yet to be established and is of importance for several applications involving Monte-Carlo approximations. We consider $L^2$ convergence in the unbounded case using a combination of traditional stochastic analysis and rough path techniques. We expect our proof technique extend to more general piecewise smooth approximations.

Stein variational gradient descent (SVGD) is a particle based approximate inference algorithm with largely well understood theoretical properties. In recent years, many variants of SVGD have been proposed and shown to share those properties. A preliminary test of the hybrid kernel variant (h-SVGD) has demonstrated promising results on image classification with deep neural network ensembles. However, the theoretical properties of h-SVGD have not yet been established, and its practical advantages have not been fully explored. In this paper, we define a hybrid kernelised Stein discrepancy (h-KSD) and prove that the h-SVGD update direction is optimal within an appropriate reproducing kernel Hilbert space. We also prove a descent lemma that guarantees a decrease in the KL divergence at each step along with other limit results. Numerical results demonstrate that h-SVGD mitigates the variance collapse behaviour of SVGD at no additional computational cost whilst remaining competitive at inference tasks.

Despite the considerable success of deep learning methods in modelling physical processes, they suffer from a variety of issues such as overfitting and lack of interpretability. In hydrology, conceptual rainfall-runoff models are simple yet fast and effective tools to represent the underlying physical processes through lumped storage components. Although conceptual rainfall-runoff models play a vital role in supporting decision-making in water resources management and urban planning, they have limited flexibility to take data into account for the development of robust region-wide models. The combination of deep learning and conceptual models has the potential to address some of the aforementioned limitations. This paper presents a sub-model hybridization of the GR4J rainfall-runoff model with deep learning architectures such as convolutional neural networks (CNN) and long short-term memory (LSTM) networks. The results show that the hybrid models outperform both the base conceptual model as well as the canonical deep neural network architectures in terms of the Nash–Sutcliffe Efficiency (NSE) score across 223 catchments in Australia. We show that our hybrid model provides a significant improvement in predictive performance, particularly in arid catchments, and generalizing better across catchments.

Model uncertainty quantification is an essential component of effective data assimilation. Model errors associated with sub-grid scale processes are often represented through stochastic parameterizations of the unresolved process. Many existing Stochastic Parameterization schemes are only applicable when knowledge of the true sub-grid scale process or full observations of the coarse scale process are available, which is typically not the case in real applications. We present a methodology for estimating the statistics of sub-grid scale processes for the more realistic case that only partial observations of the coarse scale process are available. Model error realizations are estimated over a training period by minimizing their conditional sum of squared deviations given some informative covariates (e.g., state of the system), constrained by available observations and assuming that the observation errors are smaller than the model errors. From these realizations a conditional probability distribution of additive model errors given these covariates is obtained, allowing for complex non-Gaussian error structures. Random draws from this density are then used in actual ensemble data assimilation experiments. We demonstrate the efficacy of the approach through numerical experiments with the multi-scale Lorenz 96 system using both small and large time scale separations between slow (coarse scale) and fast (fine scale) variables. The resulting error estimates and forecasts obtained with this new method are superior to those from two existing methods.

Control-type particle filters have been receiving increasing attention over the last decade as a means of obtaining sample based approximations to the sequential Bayesian filtering problem in the nonlinear setting. Here we analyse one such type, namely the feedback particle filter and a recently proposed approximation of the associated gain function based on diffusion maps. The key purpose is to provide analytic insights on the form of the approximate gain, which are of interest in their own right. These are then used to establish a roadmap to obtaining well-posedness and convergence of the finite $N$ system to its mean field limit. A number of possible future research directions are also discussed.

Various particle filters have been proposed over the last couple of decades with the common feature that the update step is governed by a type of control law. This feature makes them an attractive alternative to traditional sequential Monte Carlo which scales poorly with the state dimension due to weight degeneracy. This article proposes a unifying framework that allows to systematically derive the McKean-Vlasov representations of these filters for the discrete time and continuous time observation case, taking inspiration from the smooth approximation of the data considered in Crisan & Xiong (2010) and Clark & Crisan (2005). We consider three filters that have been proposed in the literature and use this framework to derive Ito representations of their limiting forms as the approximation parameter $\delta \rightarrow 0$. All filters require the solution of a Poisson equation defined on $\mathbb{R}^{d}$, for which existence and uniqueness of solutions can be a non-trivial issue. We additionally establish conditions on the signal-observation system that ensures well-posedness of the weighted Poisson equation arising in one of the filters.

Purpose The anatomy of the circle of Willis (CoW), the brain’s main arterial blood supply system, strongly differs between individuals, resulting in highly variable flow fields and intracranial vascularization patterns. To predict subject-specific hemodynamics with high certainty, we propose a data assimilation (DA) approach that merges fully 4D phase-contrast magnetic resonance imaging (PC-MRI) data with a numerical model in the form of computational fluid dynamics (CFD) simulations. Methods To the best of our knowledge, this study is the first to provide a transient state estimate for the three-dimensional velocity field in a subject-specific CoW geometry using DA. High-resolution velocity state estimates are obtained using the local ensemble transform Kalman filter (LETKF). Results Quantitative evaluation shows a considerable reduction (up to 90%) in the uncertainty of the velocity field state estimate after the data assimilation step. Velocity values in vessel areas that are below the resolution of the PC-MRI data (e.g., in posterior communicating arteries) are provided. Furthermore, the uncertainty of the analysis-based wall shear stress distribution is reduced by a factor of 2 for the data assimilation approach when compared to the CFD model alone. Conclusion This study demonstrates the potential of data assimilation to provide detailed information on vascular flow, and to reduce the uncertainty in such estimates by combining various sources of data in a statistically appropriate fashion.

Several numerical tools designed to overcome the challenges of smoothing in a nonlinear and non-Gaussian setting are investigated for a class of particle smoothers. The considered family of smoothers is induced by the class of linear ensemble transform filters which contains classical filters such as the stochastic ensemble Kalman filter, the ensemble square root filter, and the recently introduced nonlinear ensemble transform filter. Further the ensemble transform particle smoother is introduced and particularly highlighted as it is consistent in the particle limit and does not require assumptions with respect to the family of the posterior distribution. The linear update pattern of the considered class of linear ensemble transform smoothers allows one to implement important supplementary techniques such as adaptive spread corrections, hybrid formulations, and localization in order to facilitate their application to complex estimation problems. These additional features are derived and numerically investigated for a sequence of increasingly challenging test problems.

In this paper, we exploit the gradient flow structure of continuous-time formulations of Bayesian inference in terms of their numerical time-stepping. We focus on two particular examples, namely, the continuous-time ensemble Kalman-Bucy filter and a particle discretisation of the Fokker-Planck equation associated to Brownian dynamics. Both formulations can lead to stiff differential equations which require special numerical methods for their efficient numerical implementation. We compare discrete gradient methods to alternative semi-implicit and other iterative implementations of the underlying Bayesian inference problems.

Several numerical tools designed to overcome the challenges of smoothing in a high dimensional nonlinear setting are investigated for a class of particle smoothers. The considered family of smoothers is induced by the class of Linear Ensemble Transform Filters which contains classical filters such as the Ensemble Kalman Filter, the Ensemble Square Root Filter and the recently introduced Nonlinear Ensemble Transform Filter. Further the Ensemble Transform Particle Smoother is introduced and particularly highlighted as it is consistent in the particle limit and does not require assumptions with respect to the family of the posterior distribution. The linear update pattern of the considered class of Linear Ensemble Transform Smoothers allows one to implement important supplementary techniques such as localisation, adaptive spread corrections and hybrid formulations which combine smoothers with complementary properties. These additional features are derived and numerically investigated for the proposed family of smoothers.

The increasing availability of earth observations necessitates mathematical methods to optimally combine such data with hydrologic models. Several algorithms exist for such purposes, under the umbrella of data assimilation (DA). However, DA methods are often applied in a suboptimal fashion for complex real-world problems, due largely to several practical implementation issues. One such issue is error characterization, which is known to be critical for a successful assimilation. Mischaracterized errors lead to suboptimal forecasts, and in the worst case, to degraded estimates even compared to the no assimilation case. Model uncertainty characterization has received little attention relative to other aspects of DA science. Traditional methods rely on subjective, ad hoc tuning factors or parametric distribution assumptions that may not always be applicable. We propose a novel data-driven approach (named SDMU) to model uncertainty characterization for DA studies where (1) the system states are partially observed and (2) minimal prior knowledge of the model error processes is available, except that the errors display state dependence. It includes an approach for estimating the uncertainty in hidden model states, with the end goal of improving predictions of observed variables. The SDMU is therefore suited to DA studies where the observed variables are of primary interest. Its efficacy is demonstrated through a synthetic case study with low-dimensional chaotic dynamics and a real hydrologic experiment for one-day-ahead streamflow forecasting. In both experiments, the proposed method leads to substantial improvements in the hidden states and observed system outputs over a standard method involving perturbation with Gaussian noise.

Rapid population and economic growth in Southeast Asia has been accompanied by extensive land use change with consequent impacts on catchment hydrology. Modeling methodologies capable of handling changing land use conditions are therefore becoming ever more important and are receiving increasing attention from hydrologists. A recently developed data-assimilation-based framework that allows model parameters to vary through time in response to signals of change in observations is considered for a medium-sized catchment (2880 km2) in northern Vietnam experiencing substantial but gradual land cover change. We investigate the efficacy of the method as well as the importance of the chosen model structure in ensuring the success of a time-varying parameter method. The method was used with two lumped daily conceptual models (HBV and HyMOD) that gave good-quality streamflow predictions during pre-change conditions. Although both time-varying parameter models gave improved streamflow predictions under changed conditions compared to the time-invariant parameter model, persistent biases for low flows were apparent in the HyMOD case. It was found that HyMOD was not suited to representing the modified baseflow conditions, resulting in extreme and unrealistic time-varying parameter estimates. This work shows that the chosen model can be critical for ensuring the time-varying parameter framework successfully models streamflow under changing land cover conditions. It can also be used to determine whether land cover changes (and not just meteorological factors) contribute to the observed hydrologic changes in retrospective studies where the lack of a paired control catchment precludes such an assessment.

The global prevalence of rapid and extensive land use change necessitates hydrologic modelling methodologies capable of handling non-stationarity. This is particularly true in the context of Hydrologic Forecasting using Data Assimilation. Data Assimilation has been shown to dramatically improve forecast skill in hydrologic and meteorological applications, although such improvements are conditional on using bias-free observations and model simulations. A hydrologic model calibrated to a particular set of land cover conditions has the potential to produce biased simulations when the catchment is disturbed. This paper sheds new light on the impacts of bias or systematic errors in hydrologic data assimilation, in the context of forecasting in catchments with changing land surface conditions and a model calibrated to pre-change conditions. We posit that in such cases, the impact of systematic model errors on assimilation or forecast quality is dependent on the inherent prediction uncertainty that persists even in pre-change conditions. Through experiments on a range of catchments, we develop a conceptual relationship between total prediction uncertainty and the impacts of land cover changes on the hydrologic regime to demonstrate how forecast quality is affected when using state estimation Data Assimilation with no modifications to account for land cover changes. This work shows that systematic model errors as a result of changing or changed catchment conditions do not always necessitate adjustments to the modelling or assimilation methodology, for instance through re-calibration of the hydrologic model, time varying model parameters or revised offline/online bias estimation.

Model uncertainty quantification is an essential component of effective Data Assimilation (DA), although it can be a challenging task for complex partially observed non-linear systems with highly non-Gaussian uncertainties. Model errors associated with sub-grid scale processes are of particular interest in meteorological DA studies; these are often represented through Stochastic Parameterizations of the unresolved process. Many existing Stochastic Parameterization schemes are only applicable when knowledge of the true sub-grid scale process or full observations of the coarse scale process are available, which is typically not the case in real applications. We present a methodology for estimating the statistics of sub-grid scale processes for the more realistic case that only partial observations of the coarse scale process are available. The aim is to first estimate the conditional probability density of additive model errors given the state of the system, from which samples can be generated to simulate model error within any ensemble-based assimilation framework. Model error realizations are estimated over a training period by minimizing their conditional variance, constrained by available observations. Special is that these errors are binned conditioned on the previous model state during the minimization process, allowing for the recovery of complex non-Gaussian error structures. We demonstrate the efficacy of the approach through numerical experiments with the multi-scale Lorenz 96 system. Various parameterizations of the Lorenz 96' model are considered with both small and large time scale separations between slow (coarse scale) and fast (fine scale) variables. Results indicate that both error estimates and forecasts are improved with the proposed method compared to two existing methods for accounting for model uncertainty in DA.

We analyse various perturbations and projections of Kalman-Bucy semigroups and Riccati equations. For example, covariance inflation-type perturbations and localisation methods (projections) are common in the ensemble Kalman filtering literature. In the limit of these ensemble methods, the regularised sample covariance tends toward a solution of a perturbed/projected Riccati equation. With this motivation, results are given characterising the error between the nominal and regularised Riccati flows and Kalman-Bucy filtering distributions. New projection-type models are also discussed; e.g. Bose-Mesner projections. These regularisation models are also of interest on their own, and in, e.g., differential games, control of stochastic/jump processes, and robust control.