Inferential models (IMs) offer prior-free, Bayesian-like posterior degrees of belief designed for statistical inference, which feature a frequentist-like calibration property that ensures reliability of said inferences. The catch is that IMs' degrees of belief are possibilistic rather than probabilistic and, since the familiar Monte Carlo methods approximate probabilistic quantities, there are significant computational challenges associated with putting this framework into practice. The present article overcomes these challenges by developing a new Monte Carlo method designed specifically to approximate the IM's possibilistic output. The proposal is based on a characterization of the possibilistic IM's credal set, which identifies the "best probabilistic approximation" of the IM as a mixture distribution that can be readily approximated and sampled from. These samples can then be transformed into an approximation of the possibilistic IM. Numerical results are presented highlighting the proposed approximation's accuracy and computational efficiency. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Inferential models (IMs) are data-dependent, imprecise-probabilistic structures designed to quantify uncertainty about unknowns. As the name suggests, the focus has been on uncertainty quantification for inference and on its reliability properties in that context. Focusing on a likelihood-based possibilistic IM formulation, the present paper develops a corresponding framework for decision making, and investigates the decision-theoretic implications of the IM’s reliability guarantees. Here we show that the possibilistic IM’s assessment of an action’s quality, defined by a simple Choquet integral, tends not be too optimistic compared to that of an oracle. This ensures that the IM tends not to favor actions that the oracle doesn’t also favor, hence the IM is also reliable for decision making. We also establish a complementary, large-sample efficiency result that says the IM’s reliability isn’t achieved by being grossly conservative. In the special case of equivariant statistical models, further connections can be made between the IM’s and Bayesian’s recommended actions, from which certain optimality conclusions can be drawn.

Model selection and hypothesis testing are important tasks on networks. A key challenge lies in the inherent dependence in network data, as well as the fact that typically only a single realization is observed. As a result, many existing methods must be carefully tailored to specific models and only come with asymptotic theoretical guarantees. In this work, however, we propose a general model selection framework using Universal Inference, making our method widely applicable to various testing scenarios. Since Universal Inference requires two sets of data, we employ edge sampling to obtain proper networks with tractable dependence. We prove that the proposed statistic is an e-value, thus controlling the type I error rate in finite samples under nearly any hypothesis test. To our knowledge, this is the first Universal Inference-type statistic constructed from dependent splits of data as well as the first finite-sample testing guarantee for hypothesis testing on networks. We also prove that the logarithm of the test statistic diverges to positive infinity under various alternative models. On simulated and real-world networks, the proposed method performs well on tasks such as choosing the random graph model and the number of communities.

Compared to p-values, e-values provably guarantee safe, valid inference. Applications often require consideration of multiple hypotheses simultaneously, and tools for handling such cases using e-values can be found in the relevant literature. Standard e-value constructions, however, require distributional assumptions that may not be justifiable. This short paper demonstrates that, depending on the multiple testing context, the generalized universal inference framework is well-suited for use with the existing e-value merging and adjustment strategies to control frequentist error rates in multiple testing when the quantities of interest are minimizers of risk functions, thereby avoiding the need for certain distributional assumptions. We demonstrate the strong performance of this general approach in a simulation study involving significance testing in quantile regression.

Induction is the process by which empirical evidence is transformed to knowledge. Hume famously argued—and Popper and others agree—that there can be no logical justification for induction. A weaker form of induction, due to Bayes, expresses the aforementioned knowledge in terms of probabilities, and we review some well-known and not-so-well-known criticisms of the Bayesian solution. We then investigate the relatively new possibilistic inferential model (IM) framework, showing that, in addition to the IM's strong, statistical reliability guarantees that it uniquely enjoys, it is safe from those criticisms that damage the Bayesian foundations. For illustration, we reconsider the classical sunrise problem and compare our proposed solution with Laplace's famous rule of succession.

Community structure in networks naturally arises in various applications. But while the topic has received significant attention for static networks, the literature on community structure in temporally evolving networks is more scarce. In particular, there are currently no statistical methods available to test for the presence of community structure in a sequence of networks evolving over time. In this work, we propose a simple yet powerful test using e-values, an alternative to p-values that is more flexible in certain ways. Specifically, an e-value framework retains valid testing properties even after combining dependent information, a relevant feature in the context of testing temporal networks. We apply the proposed test to synthetic and real-world networks, demonstrating various features inherited from the e-value formulation and exposing some of the inherent difficulties of testing on temporal networks.

An inferential model (IM) is a model describing the construction of provably reliable, data-driven uncertainty quantification and inference about relevant unknowns. IMs and Fisher's fiducial argument have similar objectives, but a fundamental distinction between the two is that the former doesn't require that uncertainty quantification be probabilistic, offering greater flexibility and allowing for a proof of its reliability. Important recent developments have been made thanks in part to newfound connections with the imprecise probability literature, in particular, possibility theory. The brand of possibilistic IMs studied here are straightforward to construct, have very strong frequentist-like reliability properties, and offer fully conditional, Bayesian-like (imprecise) probabilistic reasoning. This paper reviews these key recent developments, describing the new theory, methods, and computational tools. A generalization of the basic possibilistic IM is also presented, making new and unexpected connections with ideas in modern statistics and machine learning, e.g., bootstrap and conformal prediction.

Classical statistical methods have theoretical justification when the sample size is predetermined. In applications,however, sample sizes are often data-dependent rather than predetermined. The aforementioned methods aren'treliable in this latter case, hence the recent interest in e-processes and methods that are anytime valid, i.e.,reliable for any dynamic data-collection plan. But if the investigator has relevant-yet-incomplete prior informationabout the quantity of interest, then there's an opportunity for efficiency gain. This paper proposes aregularized e-processframework featuring a knowledge-based, imprecise-probabilistic regularization with improved efficiency. Ageneralized version of Ville's inequality is established, ensuring that inference based on the regularized e-process isanytime valid in a novel, knowledge-dependent sense. Regularized e-processes also facilitate possibility-theoreticuncertainty quantification with strong frequentist-like calibration properties and other Bayesian-like properties:satisfies the likelihood principle, avoids sure-loss, and offers formal decision-making with reliability guarantees.

Inferential models (IMs) offer provably reliable, data-driven, possibilistic statistical inference. But despite the IM framework's theoretical and foundational advantages, efficient computation is a challenge. This paper presents a simple yet powerful numerical strategy for approximating the IM's possibility contour, or at least its a-cut for a specified a is an element of (0, 1). Our proposal starts with the specification of a parametric family that, in a certain sense, approximately covers the credal set associated with the IM's possibility measure. Akin to variational inference, we then propose to tune the parameters of that parametric family so that its 100(1-a)% credible set roughly matches the IM contour's a-cut. This parametric a-cut matching strategy implies a full approximation to the IM's possibility contour at a fraction of the computational cost associated with previous strategies.

Logistic regression involving high-dimensional covariates is a practically important problem. Often the goal is variable selection, i.e., determining which few of the many covariates are associated with the binary response. Unfortunately, the usual Bayesian computations can be quite challenging and expensive. Here we start with a recently proposed empirical Bayes solution, with strong theoretical convergence properties, and develop a novel and computationally efficient variational approximation thereof. One such novelty is that we develop this approximation directly for the marginal distribution on the model space, rather than on the regression coefficients themselves. We demonstrate the method's strong performance in simulations, and prove that our variational approximation inherits the strong selection consistency property satisfied by the posterior distribution that it is approximating.

Divide-and-conquer methods use large-sample approximations to provide frequentist guarantees when each block of data is both small enough to facilitate efficient computation and large enough to support approximately valid inferences. When the overall sample size is small or moderate, likely no suitable division of the data meets both requirements, hence the resulting inference lacks validity guarantees. We propose a new approach, couched in the inferential model framework, that is fully conditional in a Bayesian sense and provably valid in a frequentist sense. The main insight is that existing divide-and-conquer approaches make use of a Gaussianity assumption twice: first in the construction of an estimator, and second in the approximation to its sampling distribution. Our proposal is to retain the first Gaussianity assumption, using a Gaussian working likelihood, but to replace the second with a validification step that uses the sampling distributions of the block summaries determined by the posited model. This latter step, a type of probability-to-possibility transform, is key to the reliability guarantees enjoyed by our approach, which are uniquely general in the divide-and-conquer literature. In addition to finite-sample validity guarantees, our proposed approach is also asymptotically efficient like the other divide-and-conquer solutions available in the literature. Our computational strategy leverages state-of-the-art black-box likelihood emulators. We demonstrate our method's performance via simulations and highlight its flexibility with an analysis of median PM2.5 in Maryborough, Queensland, during the 2023 Australian bushfire season.

A common goal in statistics and machine learning is estimation of unknowns. Point estimates alone are of little value without an accompanying measure of uncertainty, but traditional uncertainty quantification methods, such as confidence sets and p-values, often require distributional or structural assumptions that may not be justified in modern applications. The present paper considers a very common case in machine learning, where the quantity of interest is the minimizer of a given risk (expected loss) function. We propose a generalization of universal inference specifically designed for inference on risk minimizers. Notably, our generalized universal inference attains finite-sample frequentist validity guarantees under a condition common in the statistical learning literature. One version of our procedure is also anytime-valid, i.e., it maintains the finite-sample validity properties regardless of the stopping rule used for the data collection process. Practical use of our proposal requires tuning, and we offer a data-driven procedure with strong empirical performance across a broad range of challenging statistical and machine learning examples.

Distinguishing two models is a fundamental and practically important statistical problem. Error rate control is crucial to the testing logic, but in complex nonparametric settings can be difficult to achieve, especially when the stopping rule that determines the data collection process is not available. This paper proposes an $ e $-process construction based on the predictive recursion algorithm originally designed to recursively fit nonparametric mixture models. The resulting predictive recursion $ e $-process affords anytime-valid inference and is asymptotically efficient in the sense that its growth rate is first-order optimal relative to the predictive recursion's mixture model.

Inference on high-dimensional parameters in structured linear models is an important statistical problem.This paper focuses on the case of a piecewise polynomial Gaussian sequence model, and we develop a new empirical Bayes solution that enjoys adaptive minimax posterior concentration rates and improved structure learning properties compared to existing methods.Moreover, thanks to the conjugate form of the empirical prior, posterior computations are fast and easy.Numerical examples also highlight the method's strong finite-sample performance compared to existing methods across a range of different scenarios.

Inferential models (IMs) are data-dependent, imprecise-probabilistic structures designed to quantify uncertainty about unknowns. As the name suggests, the focus has been on uncertainty quantification for inference and on its reliability properties in that context. The present paper develops an IM framework for decision making, and investigates the decision-theoretic implications of the IM's reliability guarantees. I show that the IM's assessment of an action's quality, defined by a Choquet integral, will not be too optimistic compared to that of an oracle. This ensures that the IM tends not to favor actions that the oracle doesn't also favor, hence a IM is reliable for decision making too. In a certain special class of structured statistical models, further connections can be made between the IM's recommended actions and those recommended by Bayesian/fiducial frameworks, from which certain optimality conclusions can be drawn.

Uncovering genuine relationships between a response variable of interest and a large collection of covariates is a fundamental and practically important problem. In the context of Gaussian linear models, both the Bayesian and non-Bayesian literature is well-developed and there are no substantial differences in the model selection consistency results available from the two schools. For the more challenging generalized linear models (GLMs), however, Bayesian model selection consistency results are lacking in several ways. In this paper, we construct a Bayesian posterior distribution using an appropriate data-dependent prior and develop its asymptotic concentration properties using new theoretical techniques. In particular, we leverage Spokoiny's powerful non-asymptotic theory to obtain sharp quadratic approximations of the GLM's log-likelihood function, which leads to tight bounds on the errors associated with the model-specific maximum likelihood estimators and the Laplace approximation of our Bayesian marginal likelihood. In turn, these improved bounds lead to significantly stronger, near-optimal Bayesian model selection consistency results, e.g., far weaker beta-min conditions, compared to those available in the existing literature. In particular, our results are applicable to the Poisson regression model, in which the score function is not sub-Gaussian.

A crucial step in fitting a regression model to data is determining the model's structure, i.e., the subset of explanatory variables to be included. However, the uncertainty in this step is often overlooked due to a lack of satisfactory methods. Frequentists have no broadly applicable confidence set constructions for a model's structure, and Bayesian posterior credible sets do not achieve the desired finite-sample coverage. In this paper, we propose an extension of the possibility-theoretic inferential model (IM) framework that offers reliable, data-driven uncertainty quantification about the unknown model structure. This particular extension allows for the inclusion of incomplete prior information about the unknown structure that facilitates regularization. We prove that this new, regularized, possibilistic IM's uncertainty quantification is suitably calibrated relative to the set of joint distributions compatible with the data-generating process and assumed partial prior knowledge about the structure. This implies, among other things, that the derived confidence sets for the unknown model structure attain the nominal coverage probability in finite samples. We provide background and guidance on quantifying prior knowledge in this new context and analyze two benchmark data sets, comparing our results to those obtained by existing methods.

Colour polymorphic populations constitute excellent model systems for testing ecological and evolutionary hypotheses, as alternate colourations are associated with differences in various biological, behavioural and life-history traits. Melanism is the most common and most obvious form of polymorphism. In reptiles in general (and snakes in particular), thermal melanism has been a popular hypothesis used to explain the appearance of black individuals. It suggests that a trade-off exists between superior thermoregulation and reduced protection through the loss of crypsis. Surprisingly, despite the growing body of literature available, to date, there are only two qualitative reviews on the evolutionary significance of melanism in reptiles and no quantitative synthesis. We conducted the first systematic review and meta-analysis summarising the current knowledge on melanism in polymorphic terrestrial snakes and synthesised the evidence for an adaptive advantage of the melanistic morph. Northern Hemisphere. Polymorphic terrestrial snakes. Mean prevalence of melanism is 31% with no significant differences between species. Annual precipitation was a significant moderator of melanism prevalence. Our results revealed no significant differences in odds ratios of melanism between sexes or in mean body size. Environmental plasticity can be considered the primary cause of melanism, possibly as a result of seasonal climatic variations, a result corroborated by the formal meta-analysis conducted. Conclusions from the meta-analysis are that melanism in snakes follows Gloger's rule, as is the case in birds and mammals, and as opposed to the thermal melanism hypothesis. Further, our findings do not lend support for other predictions from the thermal melanism hypothesis, such as skewed frequency of melanistics in favour of males or females, or larger body sizes in black individuals. Our results hold implications for the future diversity of animal populations, as climate change is predicted to decrease the degree of colour variation.

When prior information is lacking, the go-to strategy for probabilistic inference is to combine a "default prior" and the likelihood via Bayes's theorem. Objective Bayes, (generalized) fiducial inference, etc. fall under this umbrella. This construction is natural, but the corresponding posterior distributions generally only offer limited, approximately valid uncertainty quantification. The present paper takes a reimagined approach that yields posterior distributions with stronger reliability properties. The proposed construction starts with an inferential model (IM), one that takes the mathematical form of a data-driven possibility measure and features exactly valid uncertainty quantification, and then returns a so-called inner probabilistic approximation thereof. This inner probabilistic approximation inherits many of the original IM's desirable properties, including credible sets with exact coverage and asymptotic efficiency. The approximation also agrees with the familiar Bayes/fiducial solution in applications where the model has a group invariance structure. A Monte Carlo method for evaluating the probabilistic approximation is presented, along with numerical illustrations.