Abstract. This article is concerned with sampling from Gibbs distributions [Formula: see text] using Markov chain Monte Carlo methods. In particular, we investigate Langevin dynamics in the continuous- and discrete-time setting for such distributions with potentials [Formula: see text] which are strongly convex but possibly nondifferentiable and whose subgradients grow at most linearly. We show that the corresponding subgradient Langevin dynamics is exponentially ergodic with invariant distribution [Formula: see text] in the continuous setting and that its explicit discretization as well as a semi-implicit discretization are geometrically ergodic, the semi-implicit one exhibiting better convergence behavior for vanishing step sizes. We show that both discretizations approximate the target distribution [Formula: see text] for vanishing discretization step size. Moreover, we prove that the discrete schemes satisfy the law of large numbers allowing us to use consecutive iterates in order to compute statistics of the stationary distribution instead of simulating multiple independent Markov chains. Numerical experiments are provided confirming the theoretical findings and showcasing the practical relevance of the proposed methods in imaging applications.
Abstract. This paper presents sparse gradient descent as a solution for variable selection in convex piecewise linear regression where the model is given as the maximum of [Formula: see text]-affine functions [Formula: see text] for [Formula: see text]. Here, [Formula: see text] and [Formula: see text] denote the ground-truth weight vectors and intercepts. A nonasymptotic local convergence analysis is provided for Sp-GD under sub-Gaussian noise when the covariate distribution satisfies the sub-Gaussianity and anticoncentration properties. When the model order and parameters are fixed, Sp-GD provides an [Formula: see text]-accurate estimate given [Formula: see text] observations, where [Formula: see text] denotes the noise variance. This also implies the exact parameter recovery by Sp-GD from [Formula: see text] noise-free observations. Since optimizing the squared loss for sparse max-affine is nonconvex, an initialization scheme is proposed to provide a suitable initial estimate within the basin of attraction for Sp-GD, i.e., sufficiently accurate to invoke the convergence guarantees. The initialization scheme uses sparse principal component analysis to estimate the subspace spanned by [Formula: see text], then applies an [Formula: see text]-covering search to estimate the model parameters. A nonasymptotic analysis is presented for this initialization scheme when the covariates and noise samples follow Gaussian distributions. When the model order and parameters are fixed, this initialization scheme provides an [Formula: see text]-accurate estimate given [Formula: see text] observations. A new transformation named Real Maslov Dequantization (RMD) is proposed to transform sparse generalized polynomials into sparse max-affine models. The error decay rate of RMD is shown to be exponentially small in its temperature parameter. Furthermore, theoretical guarantees for Sp-GD are extended to the bounded noise model induced by RMD. Numerical Monte Carlo results corroborate theoretical findings for Sp-GD and the initialization scheme.
Abstract. This paper develops a generative model by minimizing the second-order Wasserstein loss (the [Formula: see text] loss) through a distribution-dependent ordinary differential equation (ODE), whose dynamics involves the Kantorovich potential associated with the true data distribution and a current estimate of it. A main result shows that the time-marginal laws of the ODE form a gradient flow for the [Formula: see text] loss, which converges exponentially to the true data distribution. An Euler scheme for the ODE is proposed, and it is shown to recover the gradient flow for the [Formula: see text] loss in the limit. An algorithm is designed by following the scheme and applying persistent training, which naturally fits our gradient-flow approach. In both low- and high-dimensional experiments, our algorithm outperforms Wasserstein generative adversarial networks by increasing the level of persistent training appropriately.
Abstract. We propose a new notion of uniqueness for the adversarial Bayes classifier in the setting of binary classification. Analyzing this concept produces a simple procedure for computing all adversarial Bayes classifiers for a well-motivated family of one-dimensional data distributions. This characterization is then leveraged to show that as the perturbation radius increases, certain notions of regularity of adversarial Bayes classifiers improve. These results provide tools for understanding relationships between the Bayes and adversarial Bayes classifiers in one dimension.
Continuous-depth graph neural networks, also known as Graph Neural Differential Equations (GNDEs), combine the structural inductive bias of Graph Neural Networks (GNNs) with the continuous-depth architecture of Neural ODEs, offering a scalable and principled framework for modeling dynamics on graphs. In this paper, we present a rigorous convergence analysis of GNDEs with time-varying parameters in the infinite-node limit, providing theoretical insights into their size transferability. To this end, we introduce Graphon Neural Differential Equations (Graphon-NDEs) as the infinite-node limit of GNDEs and establish their well-posedness. Leveraging tools from graphon theory and dynamical systems, we prove the trajectory-wise convergence of GNDE solutions to Graphon-NDE solutions. Moreover, we derive explicit convergence rates under two deterministic graph sampling regimes: (1) weighted graphs sampled from smooth graphons, and (2) unweighted graphs sampled from $\{0,1\}$-valued (discontinuous) graphons. We further establish size transferability bounds, providing theoretical justification for the practical strategy of transferring GNDE models trained on moderate-sized graphs to larger, structurally similar graphs without retraining. Numerical experiments using synthetic and real data support our theoretical findings.
