Robustness of neural networks is commonly quantified via local or global Lipschitz constants. However, Lipschitz continuity can be overly coarse or overly restrictive as global robustness measure, failing to capture nuanced, data-dependent behavior. We propose a data-driven, architecture-agnostic framework based on the discrete modulus of continuity (DMOC), a non linear generalization of Lipschitz continuity that provides a finer notion of robustness. Unlike many existing approaches, DMOC does not require access to model internals and instead evaluates regularity relative to the data distribution. This shifts the focus from the model to the data, which provide a data-driven baseline of regularity against which the network's robustness is assessed. We establish convergence results for DMOC-induced seminorms with explicit data-driven rates in terms of the separation distance, and introduce a scalable minibatch algorithm that reduces the quadratic cost of exact computation, enabling application to large-scale data sets such as ImageNet. Empirically, DMOC serves as an architecture independent diagnostic: it distinguishes trained from untrained networks, reveals underfitting and overfitting regimes, and yields, as a special case, tight Lipschitz estimates comparable to state-of-the-art method such as ECLipsE and ECLipsE-fast.

We develop a novel framework for sparse multiscale kernel approximation of large scattered data problems based on a samplet representation. Samplets form a multiresolution analysis of localized discrete signed measures and enable quasi-sparse representations of kernel matrices associated to asymptotically smooth kernels as well as smoothness detection of scattered data. Building on the latter, we introduce an adaptive data site selection strategy based on the localization of the native reproducing kernel Hilbert space norm in the samplet expansion coefficients. The selection results in a small set of representative data sites, significantly reducing the effective problem size. On the corresponding reduced kernel subspace, we solve an ℓ^1-regularized least-squares problem using a trust-region semismooth Newton method in a normal-map formulation, stabilized by an online low-rank SVD on the active set to handle the notorious ill-conditioning of kernel matrices. Numerical experiments in two and three dimensions, including multi-kernel models with varying lengthscales, demonstrate that the proposed approach achieves accurate reconstructions with considerably sparser representations and good computational efficiency.

We consider scattered data approximation on product regions of equal and different dimensionality. On each of these regions, we assume quasi-uniform but unstructured data sites and construct optimal sparse grids for scattered data interpolation on the product region. For this, we derive new improved error estimates for the respective kernel interpolation error by invoking duality arguments. An efficient algorithm to solve the underlying linear system of equations is proposed. The algorithm is based on the sparse grid combination technique, where a sparse direct solver is used for the elementary anisotropic tensor product kernel interpolation problems. The application of the sparse direct solver is facilitated by applying a samplet matrix compression to each univariate kernel matrix, resulting in an essentially sparse representation of the latter. In this way, we obtain a method that is able to deal with large problems up to billions of interpolation points, especially in case of reproducing kernels of nonlocal nature. Numerical results are presented to qualify and quantify the approach.

We propose a scalable and theoretically grounded low-rank conditional expectation model for recursive Monte Carlo optimal stopping problems, in particular American option pricing. Our method reformulates the estimation of continuation values as a learning problem in a reproducing kernel Hilbert space, in which the conditional expectation is represented as a linear operator acting on future payoffs. This perspective yields an offline-online decomposition: the operator is learned once from simulated data and subsequently reused across all exercise dates, eliminating the need to recompute regression models at each step of the backward recursion. We establish convergence guarantees and derive bounds quantifying the approximation errors across exercise dates. Numerical experiments demonstrate the speed and accuracy of the proposed approach relative to extant methods.

The present article is concerned scattered data approximation for higher dimensional data sets which exhibit an anisotropic behavior in the different dimensions. Tailoring sparse polynomial interpolation to this specific situation, we derive very efficient degenerate kernel approximations which we then use in a dimension weighted fast multipole method. This dimension weighted fast multipole method enables to deal with many more dimensions than the standard black-box fast multipole method based on interpolation. A thorough analysis of the method is provided including rigorous error estimates. The accuracy and the cost of the approach are validated by extensive numerical results. As a relevant application, we apply the approach to a shape uncertainty quantification problem.

A straightforward way to represent the kernel approximant of a function, known by a finite set of samples, within a reproducing kernel Hilbert space is through the canonical dual pair. The canonical dual pair consists of the basis of kernel translates and the corresponding Lagrange basis. From a numerical perspective, one is particularly interested in dual pairs such that the dual basis is quasi-local meaning that it can be well approximated using only a small subset of the data sites. This implies that the inverse Gramian is approximately sparse. In this case, the kernel approximant is efficiently computable by multiplying a sparse matrix with the data vector. We present two methods for finding such quasi-localized dual bases. First, we adapt the idea of localizing the Lagrange basis, which yields an approximate canonical dual pair and extend this idea to derive a new, symmetric preconditioner for kernel matrices. Second, we use samplets to obtain multiresolution versions of dual bases. Samplets are localized discrete signed measures constructed such that their respective measure integrals of polynomials up to a certain degree vanish. Therefore, the kernel matrix and its inverse are compressible to sparse matrices in samplet coordinates for asymptotically smooth kernels. We provide benchmark experiments in two spatial dimensions to demonstrate the compression power of both approaches and apply the new preconditioner to implicit surface reconstruction in computer graphics.

Analysis and processing of data is a vital part of our modern society and requires vast amounts of computational resources. To reduce the computational burden, compressing and approximating data has become a central topic. We consider the approximation of labeled data samples, mathematically described as site-to-value maps between finite metric spaces. Within this setting, we identify the discrete modulus of continuity as an effective data-intrinsic quantity to measure regularity of site-to-value maps without imposing further structural assumptions. We investigate the consistency of the discrete modulus of continuity in the infinite data limit and propose an algorithm for its efficient computation. Building on these results, we present a sample based approximation theory for labeled data. For data subject to statistical uncertainty we consider multilevel approximation spaces and a variant of the multilevel Monte Carlo method to compute statistical quantities of interest. Our considerations connect approximation theory for labeled data in metric spaces to the covering problem for (random) balls on the one hand and the efficient evaluation of the discrete modulus of continuity to combinatorial optimization on the other hand. We provide extensive numerical studies to illustrate the feasibility of the approach and to validate our theoretical results.

We develop a new framework for embedding joint probability distributions in tensor product reproducing kernel Hilbert spaces (RKHS). Our framework accommodates a low-dimensional, normalized and positive model of a Radon-Nikodym derivative, which we estimate from sample sizes of up to several million data points, alleviating the inherent limitations of RKHS modeling. Well-defined normalized and positive conditional distributions are natural by-products to our approach. The embedding is fast to compute and accommodates learning problems ranging from prediction to classification. Our theoretical findings are supplemented by favorable numerical results.

Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities at the forefront of image, video, and audio synthesis. However, they can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. The threat posed by GANs has sparked the need to discern between genuine content and fabricated one. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces, which unfortunately did not prove very effective. Subsequent methods have focused on leveraging on frequency decompositions, i.e., discrete cosine transform, wavelets, and wavelet packets, to preprocess the input features for classifiers. However, existing approaches only rely on isotropic transformations. We argue that, since GANs primarily utilize isotropic convolutions to generate their output, they leave clear traces, their fingerprint, in the coefficient distribution on sub-bands extracted by anisotropic transformations. We employ the fully separable wavelet transform and multiwavelets to obtain the anisotropic features to feed to standard CNN classifiers. Lastly, we find the fully separable transform capable of improving the state-of-the-art.

We study time harmonic acoustic scattering on large deviation rough random scatterers. Therein, the roughness of the scatterers is caused by a low Sobolev regularity in the covariance function of their deformation field. The motivation for this study arises from physical phenomena where small-scale material defects can potentially introduce non-smooth deviations from a reference domain. The primary challenge in this scenario is that the scattered wave is also random, which makes computational predictions unreliable. Therefore, it is essential to quantify these uncertainties to ensure robust and well-informed design processes. While existing methods for uncertainty quantification typically rely on domain mapping or perturbation approaches, it turns out that large and rough random deviations are not satisfactory covered. To close this gap, and although counter intuitive at first, we show that the p-multilevel Monte Carlo method can provide an efficient tool for uncertainty quantification in this setting. To this end, we discuss the stable implementation of higher-order polynomial approximation of the deformation field by means of barycentric interpolation and provide a cost-to-accuracy analysis. Our considerations are complemented by numerical experiments in three dimensions on a complex scattering geometry.

We study multiscale scattered data interpolation schemes for globally supported radial basis functions with focus on the Matern class. The multiscale approximation is constructed through a sequence of residual corrections, where radial basis functions with different lengthscale parameters are combined to capture varying levels of detail. We prove that the condition numbers of the diagonal blocks of the corresponding multiscale system remain bounded independently of the particular level, allowing us to use an iterative solver with a bounded number of iterations for the numerical solution. Employing an appropriate diagonal scaling, the multiscale system becomes well-conditioned. We exploit this fact to derive a general error estimate bounding the consistency error issuing from a numerical approximation of the multiscale system. To apply the multiscale approach to large data sets, we suggest representing each level of the multiscale system in samplet coordinates. Samplets are localized, discrete signed measures exhibiting vanishing moments and allow for the sparse approximation of generalized Vandermonde matrices issuing from a vast class of radial basis functions. Given a quasi-uniform set of N data sites, and local approximation spaces with exponentially decreasing dimension, the samplet compressed multiscale system can be assembled with cost O (N log(2) N). The overall cost of the proposed approach is O (N log(2) N). The theoretical findings are accompanied by extensive numerical studies in two and three spatial dimensions.

In scattered data approximation, the span of a finite number of translates of a chosen radial basis function is used as approximation space and the basis of translates is used for representing the approximate. However, this natural choice is by no means mandatory and different choices, like, for example, the Lagrange basis, are possible and might offer additional features. In this article, we discuss different alternatives together with their canonical duals. We study a localized version of the Lagrange basis, localized orthogonal bases, such as the Newton basis, and multiresolution versions thereof, constructed by means of samplets. We argue that the choice of orthogonal bases is particularly useful as they lead to symmetric preconditioners. All bases under consideration are compared numerically to illustrate their feasibility for scattered data approximation. We provide benchmark experiments in two spatial dimensions and consider the reconstruction of an implicit surface as a relevant application from computer graphics.

We seek to extract a small number of representative scenarios from large and high-dimensional panel data that are consistent with sample moments. Among two novel algorithms, the first identifies scenarios that have not been observed before, and comes with a scenario-based representation of covariance matrices. The second proposal picks important data points from states of the world that have already realized, and are consistent with higher-order sample moment information. Both algorithms are efficient to compute, and lend themselves to consistent scenario-based modeling and high-dimensional numerical integration. Extensive numerical benchmarking studies and an application in portfolio optimization favor the proposed algorithms.

This work introduces the definition of observation-specific explanations to assign a score to each data point proportional to its importance in the definition of the prediction process. Such explanations involve the identification of the most influential observations for the black-box model of interest. The proposed method involves estimating these explanations by constructing a surrogate model through scattered data approximation utilizing the orthogonal matching pursuit algorithm. The proposed approach is validated on both simulated and real-world datasets.

We propose a sparse algebra for samplet compressed kernel matrices to enable efficient scattered data analysis. We show that the compression of kernel matrices by means of samplets produces optimally sparse matrices in a certain S-format. The compression can be performed in cost and memory that scale essentially linearly with the number of data points for kernels of finite differentiability. The same holds true for the addition and multiplication of S-formatted matrices. We prove that the inverse of a kernel matrix, given that it exists, is compressible in the S-format as well. The use of selected inversion allows to directly compute the entries in the corresponding sparsity pattern. Moreover, S-formatted matrix operations enable the efficient, approximate computation of more complicated matrix functions such as A^α or exp (A) of a matrix A . The matrix algebra is justified mathematically by pseudo differential calculus. As an application, we consider Gaussian process learning algorithms for implicit surfaces. Numerical results are presented to illustrate and quantify our findings.

We solve acoustic scattering problems by means of the isogeometric boundary integral equation method. In order to avoid spurious modes, we apply the combined field integral equations for either sound-hard scatterers or sound-soft scatterers. These integral equations are discretized by Galerkin’s method, which especially enables the mathematically correct regularization of the hypersingular integral operator. In order to circumvent densely populated system matrices, we employ the isogeometric embedded fast multipole method, which is based on interpolation of the kernel function under consideration on the reference domain, rather than in space. To overcome the prohibitive cost of the potential evaluation in case of many evaluation points, we also accelerate the potential evaluation by a fast multipole method which interpolates in space. The result is a frequency stable algorithm that scales essentially linear in the number of degrees of freedom and potential points. Numerical experiments are performed which show the feasibility and the performance of the approach.

This chapter is dedicated to recent developments in the field of wavelet analysis for scattered data. We introduce the concept of samplets, which are signed measures of wavelet type and may be defined on sets of arbitrarily distributed data sites in possibly high dimension. By employing samplets, we transfer well-known concepts known from wavelet analysis, namely the fast basis transform, data compression, operator compression and operator arithmetics to scattered data problems. Especially, samplet matrix compression facilitates the rapid solution of scattered data interpolation problems, even for kernel functions with nonlocal support. Finally, we demonstrate that sparsity constraints for scattered data approximation problems become meaningful and can efficiently be solved in samplet coordinates.

We consider scattered data approximation in samplet coordinates with $\ell_{1}$-regularization. The application of an $\ell_{1}$-regularization term enforces sparsity of the coefficients with respect to the samplet basis. Samplets are wavelet-type signed measures, which are tailored to scattered data. Therefore, samplets enable the use of well-established multiresolution techniques on general scattered data sets. They provide similar properties as wavelets in terms of localization, multiresolution analysis, and data compression. By using the Riesz isometry, we embed samplets into reproducing kernel Hilbert spaces and discuss the properties of the resulting functions. We argue that the class of signals that are sparse with respect to the embedded samplet basis is considerably larger than the class of signals that are sparse with respect to the basis of kernel translates. Vice versa, every signal that is a linear combination of only a few kernel translates is sparse in samplet coordinates. We propose the rapid solution of the problem under consideration by combining soft-shrinkage with the semi-smooth Newton method. Leveraging on the sparse representation of kernel matrices in samplet coordinates, this approach converges faster than the fast iterative shrinkage thresholding algorithm and is feasible for large-scale data. Numerical benchmarks are presented and demonstrate the superiority of the multiresolution approach over the single-scale approach. As large-scale applications, the surface reconstruction from scattered data and the reconstruction of scattered temperature data using a dictionary of multiple kernels are considered.

We consider scattered data approximation in samplet coordinates with $\ell_1$-regularization. The application of an $\ell_1$-regularization term enforces sparsity of the coefficients with respect to the samplet basis. Samplets are wavelet-type signed measures, which are tailored to scattered data. Therefore, samplets enable the use of well-established multiresolution techniques on general scattered data sets. They provide similar properties as wavelets in terms of localization, multiresolution analysis, and data compression. By using the Riesz isometry, we embed samplets into reproducing kernel Hilbert spaces and discuss the properties of the resulting functions. We argue that the class of signals that are sparse with respect to the embedded samplet basis is considerably larger than the class of signals that are sparse with respect to the basis of kernel translates. Vice versa, every signal that is a linear combination of only a few kernel translates is sparse in samplet coordinates. We propose the rapid solution of the problem under consideration by combining soft-shrinkage with the semi-smooth Newton method. Leveraging on the sparse representation of kernel matrices in samplet coordinates, this approach converges faster than the fast iterative shrinkage thresholding algorithm and is feasible for large-scale data. Numerical benchmarks are presented and demonstrate the superiority of the multiresolution approach over the single-scale approach. As large-scale applications, the surface reconstruction from scattered data and the reconstruction of scattered temperature data using a dictionary of multiple kernels are considered.

We present a novel approach which aims at high-performance uncertainty quantification for cardiac electrophysiology simulations. Employing the monodomain equation to model the transmembrane potential inside the cardiac cells, we evaluate the effect of spatially correlated perturbations of the heart fibers on the statistics of the resulting quantities of interest. Our methodology relies on a close integration of multilevel quadrature methods, parallel iterative solvers, and space-time finite element discretizations, allowing for a fully parallelized framework in space, time, and stochastics. Extensive numerical studies are presented to evaluate convergence rates and to compare the performance of classical Monte Carlo methods such as standard Monte Carlo (MC) and quasi-Monte Carlo (QMC), as well as multilevel strategies, i.e., multilevel Monte Carlo (MLMC) and multilevel quasi-Monte Carlo (MLQMC) on hierarchies of nested meshes. We especially also employ a recently suggested variant of the multilevel approach for nonnested meshes to deal with a realistic heart geometry.