The automated collection of streaming observational data has become standard and defies most traditional analytic techniques. It is not just that models are hard to identify, there may not be any model that can be safely and usefully assumed. Indeed, frequently it is only predictions that can be made and assessed. Problems for this kind of data are often called & Mscr;-Open and have motivated new statistical approaches and philosophies. This paper will review some of the most successful recent predictive methods for the & Mscr;-Open problem class. Techniques include predictors from Bayesian nonparametrics such as Gaussian process priors, predictors using expert advice such as the Shtarkov solution, hash function based predictors such as the Count-Min sketch, conformal prediction, and neural nets including Long Short-Term Memory networks.
Blind source separation (BSS), particularly independent component analysis (ICA), has been widely used in various fields of science such as biomedical signal processing to recover latent source signals from the observed mixture. While ICA is typically applied to individual datasets, many real-world applications share underlying sources across datasets. Independent vector analysis (IVA) extends ICA to jointly analyze multiple datasets by exploiting statistical dependencies across them. While various IVA methods have been presented in signal processing literature, the statistical properties of methods remains largely unexplored. This article introduces the IVA model, numerous density models used in IVA, and various classical IVA methods to statistics community highlighting the need for further theoretical developments.
This article surveys a flexible and unifying framework for constructing locally adaptive Hamiltonian Monte Carlo (HMC) samplers via Gibbs sampling of tuning parameters - a strategy we refer to as Gibbs self-tuning (GIST). The core idea is to treat algorithmic parameters, such as path length, as auxiliary variables and sample them conditionally based on the current position and momentum, with reversibility ensured through a measure-preserving involution within a Metropolis-adjusted scheme. This perspective both generalizes and connects several prominent adaptive HMC methods, including randomized HMC, multinomial HMC, the No-U-Turn Sampler (NUTS), and the Apogee-to-Apogee Path Sampler, all of which are special cases of the GIST framework. As a concrete example, we present the one-sided No-U-Turn Sampler, a simple alternative to NUTS that demonstrates how GIST can be used to design new adaptive HMC algorithms. We evaluate this variant empirically across a diverse suite of models. By highlighting shared structure and theoretical foundations, this survey aims to clarify the landscape of locally adaptive HMC algorithms and point toward principled directions for future development.
Models with dimension more than the available sample size are now commonly used in various applications. A sensible inference is possible using a lower-dimensional structure. In regression problems with a large number of predictors, the model is often assumed to be sparse, with only a few predictors active. Interdependence between a large number of variables is succinctly described by a graphical model, where variables are represented by nodes on a graph and an edge between two nodes is used to indicate their conditional dependence given other variables. Many procedures for making inferences in the high-dimensional setting, typically using penalty functions to induce sparsity in the solution obtained by minimizing a loss function, were developed. Bayesian methods have been proposed for such problems more recently, where the prior takes care of the sparsity structure. These methods have the natural ability to also automatically quantify the uncertainty of the inference through the posterior distribution. Theoretical studies of Bayesian procedures in high-dimension have been carried out recently. Questions that arise are, whether the posterior distribution contracts near the true value of the parameter at the minimax optimal rate, whether the correct lower-dimensional structure is discovered with high posterior probability, and whether a credible region has adequate frequentist coverage. In this paper, we review these properties of Bayesian and related methods for several high-dimensional models such as many normal means problem, linear regression, generalized linear models, Gaussian and non-Gaussian graphical models. Effective computational approaches are also discussed.
This is an expository article on the score-based diffusion models, with a particular focus on the formulation via stochastic differential equations (SDE). After a gentle introduction, we discuss the two pillars in the diffusion modeling-sampling and score matching, which encompass the SDE/ODE sampling, score matching efficiency, the consistency models, and reinforcement learning. Short proofs are given to illustrate the main idea of the stated results. The article is primarily a technical introduction to the field, and practitioners may also find some analysis useful in designing new models or algorithms.
