We propose a constructive approach to building temporal point processes that incorporate dependence on their history. The dependence is modeled through the conditional density of the duration, i.e., the interval between successive event times, using a mixture of first-order conditional densities for each one of a specific number of lagged durations. Such a formulation for the conditional duration density accommodates high-order dynamics, and it thus enables flexible modeling for point processes with memory. The implied conditional intensity function admits a representation as a local mixture of first-order hazard functions. By specifying appropriate families of distributions for the first-order conditional densities, with different shapes for the associated hazard functions, we can obtain either self-exciting or self-regulating point processes. From the perspective of duration processes, we develop a method to specify a stationary marginal density. The resulting model, interpreted as a dependent renewal process, introduces high-order Markov dependence among identically distributed durations. Furthermore, we provide extensions to cluster point processes. These can describe duration clustering behaviors attributed to different factors, thus expanding the scope of the modeling framework to a wider range of applications. Regarding implementation, we develop a Bayesian approach to inference, model checking, and prediction. We investigate point process model properties analytically, and illustrate the methodology with both synthetic and real data examples.

We develop a nonparametric Bayesian modeling approach to ordinal regression based on priors placed directly on the discrete distribution of the ordinal responses. The prior probability models are built from a structured mixture of multinomial distributions. We leverage a continuation-ratio logits representation to formulate the mixture kernel, with mixture weights defined through the logit stick-breaking process that incorporates the covariates through a linear function. The implied regression functions for the response probabilities can be expressed as weighted sums of parametric regression functions, with covariate-dependent weights. Thus, the modeling approach achieves flexible ordinal regression relationships, avoiding linearity or additivity assumptions in the covariate effects. Model flexibility is formally explored through the Kullback-Leibler support of the prior probability model. A key model feature is that the parameters for both the mixture kernel and the mixture weights can be associated with a continuation-ratio logits regression structure. Hence, an efficient and relatively easy to implement posterior simulation method can be designed, using Pólya-Gamma data augmentation. Moreover, the model is built from a conditional independence structure for category-specific parameters, which results in additional computational efficiency gains through partial parallel sampling. In addition to the general mixture structure, we study simplified model versions that incorporate covariate dependence only in the mixture kernel parameters or only in the mixture weights. For all proposed models, we discuss approaches to prior specification and develop Markov chain Monte Carlo methods for posterior simulation. The methodology is illustrated with several synthetic and real data examples.

We propose a new family of error distributions for model-based quantile regression, which is constructed through a structured mixture of normal distributions. The construction enables fixing specific percentiles of the distribution while, at the same time, allowing for varying mode, skewness and tail behavior. It thus overcomes the severe limitation of the asymmetric Laplace distribution -- the most commonly used error model for parametric quantile regression -- for which the skewness of the error density is fully specified when a particular percentile is fixed. We develop a Bayesian formulation for the proposed quantile regression model, including conditional lasso regularized quantile regression based on a hierarchical Laplace prior for the regression coefficients, and a Tobit quantile regression model. Posterior inference is implemented via Markov Chain Monte Carlo methods. The flexibility of the new model relative to the asymmetric Laplace distribution is studied through relevant model properties, and through a simulation experiment to compare the two error distributions in regularized quantile regression. Moreover, model performance in linear quantile regression, regularized quantile regression, and Tobit quantile regression is illustrated with data examples that have been previously considered in the literature.

Development of a flexible Erlang mixture model for survival analysis is introduced. The model for the survival density is built from a structured mixture of Erlang densities, mixing on the integer shape parameter with a common scale parameter. The mixture weights are constructed through increments of a distribution function on the positive real line, which is assigned a Dirichlet process prior. The model has a relatively simple structure, balancing flexibility with efficient posterior computation. Moreover, it implies a mixture representation for the hazard function that involves time-dependent mixture weights, thus offering a general approach to hazard estimation. Extension of the model is made to accommodate survival responses corresponding to multiple experimental groups, using a dependent Dirichlet process prior for the group-specific distributions that define the mixture weights. Model properties, prior specification, and posterior simulation are discussed, and the methodology is illustrated with synthetic and real data examples.

Longitudinal studies with binary or ordinal responses are widely encountered in various disciplines, where the primary focus is on the temporal evolution of the probability of each response category. Traditional approaches build from the generalized mixed effects modeling framework. Even amplified with nonparametric priors placed on the fixed or random effects, such models are restrictive due to the implied assumptions on the marginal expectation and covariance structure of the responses. We tackle the problem from a functional data analysis perspective, treating the observations for each subject as realizations from subject-specific stochastic processes at the measured times. We develop the methodology focusing initially on binary responses, for which we assume the stochastic processes have Binomial marginal distributions. Leveraging the logits representation, we model the discrete space processes through sequences of continuous space processes. We utilize a hierarchical framework to model the mean and covariance kernel of the continuous space processes nonparametrically and simultaneously through a Gaussian process prior and an Inverse-Wishart process prior, respectively. The prior structure results in flexible inference for the evolution and correlation of binary responses, while allowing for borrowing of strength across all subjects. The modeling approach can be naturally extended to ordinal responses. Here, the continuation-ratio logits factorization of the multinomial distribution is key for efficient modeling and inference, including a practical way of dealing with unbalanced longitudinal data. The methodology is illustrated with synthetic data examples and an analysis of college students' mental health status data.

We develop a nonparametric Bayesian modeling framework for clustered ordinal responses in developmental toxicity studies, which typically exhibit extensive heterogeneity. The primary focus of these studies is to examine the dose-response relationship, which is depicted by the (conditional) probability of an endpoint across the dose (toxin) levels. Standard parametric approaches, limited in terms of the response distribution and/or the dose-response relationship, hinder reliable uncertainty quantification in this context. We propose nonparametric mixture models that are built from dose-dependent stick-breaking process priors, leveraging the continuation-ratio logits representation of the multinomial distribution to formulate the mixture kernel. We further elaborate the modeling approach, amplifying the mixture models with an overdispersed kernel which offers enhanced control of variability. We conduct a simulation study to demonstrate the benefits of both the discrete nonparametric mixing structure and the overdispersed kernel in delivering coherent uncertainty quantification. Further illustration is provided with different forms of risk assessment, using data from a toxicity experiment on the effects of ethylene glycol.

We develop a class of nearest-neighbor mixture models that provide direct, computationally efficient, probabilistic modeling for non-Gaussian geospatial data. The class is defined over a directed acyclic graph, which implies conditional independence in representing a multivariate distribution through factorization into a product of univariate conditionals, and is extended to a full spatial process. We model each conditional as a mixture of spatially varying transition kernels, with locally adaptive weights, for each one of a given number of nearest neighbors. The modeling framework emphasizes the description of non-Gaussian dependence at the data level, in contrast with approaches that introduce a spatial process for transformed data, or for functionals of the data probability distribution. Thus, it facilitates efficient, full simulation-based inference. We study model construction and properties analytically through specification of bivariate distributions that define the local transition kernels, providing a general strategy for modeling general types of non-Gaussian data. Regarding computation, the framework lays out a new approach to handling spatial data sets, leveraging a mixture model structure to avoid computational issues that arise from large matrix operations. We illustrate the methodology using synthetic data examples and an analysis of Mediterranean Sea surface temperature observations.

We introduce a flexible and scalable class of Bayesian geostatistical models for discrete data, based on nearest‐neighbor mixture processes (NNMP), referred to as discrete NNMP. To define the joint probability mass function (pmf) over a set of spatial locations, we build from local mixtures of conditional pmfs using a directed graphical model, with a directed acyclic graph that summarizes the nearest neighbor structure. The approach supports direct, flexible modeling for multivariate dependence through specification of general bivariate discrete distributions that define the conditional pmfs. In particular, we develop a modeling and inferential framework for copula‐based NNMPs that can attain flexible dependence structures, motivating the use of bivariate copula families for spatial processes. Moreover, the framework allows for construction of models given a pre‐specified family of marginal distributions that can vary in space, facilitating covariate inclusion. Compared to the traditional class of spatial generalized linear mixed models, where spatial dependence is introduced through a transformation of response means, our process‐based modeling approach provides both computational and inferential advantages. We illustrate the methodology with synthetic data examples and an analysis of North American Breeding Bird Survey data.

We develop a mixture model for transition density approximation, together with soft model selection, in the presence of noisy and heterogeneous nonlinear dynamics. Our model builds on the Gaussian mixture transition distribution (MTD) model for continuous state spaces, extending component means with nonlinear functions that are modeled using Gaussian process (GP) priors. The resulting model flexibly captures nonlinear and heterogeneous lag dependence when several mixture components are active, identifies low-order nonlinear dependence while inferring relevant lags when few components are active, and averages over multiple and competing single-lag models to quantify/propagate uncertainty. Sparsity-inducing priors on the mixture weights aid in selecting a subset of active lags. The hierarchical model specification follows conventions for both GP regression and MTD models, admitting a convenient Gibbs sampling scheme for posterior inference. We demonstrate properties of the proposed model with two simulated and two real time series, emphasizing approximation of lag-dependent transition densities and model selection. In most cases, the model decisively recovers important features. The proposed model provides a simple, yet flexible framework that preserves useful and distinguishing characteristics of the MTD model class.

We develop a Bayesian nonparametric autoregressive model applied to flexibly estimate general transition densities exhibiting nonlinear lag dependence. Our approach is related to Bayesian density regression using Dirichlet process mixtures, with the Markovian likelihood defined through the conditional distribution obtained from the mixture. This results in a Bayesian nonparametric extension of a mixtures-of-experts model formulation. We address computational challenges to posterior sampling that arise from the Markovian structure in the likelihood. The base model is illustrated with synthetic data from a classical model for population dynamics, as well as a series of waiting times between eruptions of Old Faithful Geyser. We study inferences available through the base model before extending the methodology to include automatic relevance detection among a pre-specified set of lags. Inference for global and local lag selection is explored with additional simulation studies, and the methods are illustrated through analysis of an annual time series of pink salmon abundance in a stream in Alaska. We further explore and compare transition density estimation performance for alternative configurations of the proposed model. Supplementary materials are available online.

We develop a mixture model and diagnostic for Bayesian estimation and selection in high-order, discrete-state Markov chains. Both extend the mixture transition distribution, which constructs a transition probability tensor by aggregating probabilities from a set of single-lag transition matrices, through inclusion of mixture components dependent on multiple lags. We demonstrate two uses for the proposed model: identification of relevant lags through over-specification and shrinkage via priors for sparse probability vectors, and parsimonious approximation of multi-lag dynamics by mixing low-order transition models. The diagnostic yields a general and interpretable mixture decomposition for transition probability tensors estimated by any means. We demonstrate the utility of the model and diagnostic with simulation studies, and further apply the methodology to a data analysis from the high-order Markov chain literature, and to a time series of pink salmon abundance in Alaska, United States. Supplemental files for this article are available online.

Mixture transition distribution (MTD) time series models build high-order dependence through a weighted combination of first-order transition densities for each one of a specified number of lags. We present a framework to construct stationary MTD models that extend beyond linear, Gaussian dynamics. We study conditions for first-order strict stationarity which allow for different constructions with either continuous or discrete families for the first-order transition densities given a prespecified family for the marginal density, and with general forms for the resulting conditional expectations. Inference and prediction are developed under the Bayesian framework with particular emphasis on flexible, structured priors for the mixture weights. Model properties are investigated both analytically and through synthetic data examples. Finally, Poisson and Lomax examples are illustrated through real data applications. Supplementary files for this article are available online.

We develop a class of nearest neighbor mixture transition distribution process (NNMP) models that provides flexibility and scalability for non-Gaussian geostatistical data. We use a directed acyclic graph to define a proper spatial process with finite-dimensional distributions given by finite mixtures. We develop conditions to construct general NNMP models with pre-specified stationary marginal distributions. We also establish lower bounds for the strength of the tail dependence implied by NNMP models, demonstrating the flexibility of the proposed methodology for modeling multivariate dependence through bivariate distribution specification. To implement inference and prediction, we formulate a Bayesian hierarchical model for the data, using the NNMP prior model for the spatial random effects process. From an inferential point of view, the NNMP model lays out a new computational approach to handling large spatial data sets, leveraging the mixture model structure to avoid computational issues that arise from large matrix operations. We illustrate the benefits of the NNMP modeling framework using synthetic data examples and through analysis of sea surface temperature data from the Mediterranean sea.

We develop nonparametric Bayesian modelling approaches for Poisson processes, using weighted combinations of structured beta densities to represent the point process intensity function. For a regular spatial domain, such as the unit square, the model construction implies a Bernstein-Dirichlet prior for the Poisson process density, which supports general inference for point process functionals. The key contribution of the methodology is two classes of flexible and computationally efficient models for spatial Poisson process intensities over irregular domains. We address the choice or estimation of the number of beta basis densities, and develop methods for prior specification and posterior simulation for full inference about functionals of the point process. The methodology is illustrated with both synthetic and real data sets.

We develop a prior probability model for temporal Poisson process intensities through structured mixtures of Erlang densities with common scale parameter, mixing on the integer shape parameters. The mixture weights are constructed through increments of a cumulative intensity function which is modeled nonparametrically with a gamma process prior. Such model specification provides a novel extension of Erlang mixtures for density estimation to the intensity estimation setting. The prior model structure supports general shapes for the point process intensity function, and it also enables effective handling of the Poisson process likelihood normalizing term resulting in efficient posterior simulation. The Erlang mixture modeling approach is further elaborated to develop an inference method for spatial Poisson processes. The methodology is examined relative to existing Bayesian nonparametric modeling approaches, including empirical comparison with Gaussian process prior based models, and is illustrated with synthetic and real data examples.

SummaryAn integro-difference equation can be represented as a hierarchical spatiotemporal dynamic model using appropriate parameterizations. The dynamics of the process defined by an integro-difference equation depends on the choice of a bivariate kernel distribution, where more flexible shapes generally result in more flexible models. Under a Bayesian modelling framework, we consider the use of the stable family of distributions for the kernel, as they are infinitely divisible and offer a variety of tail behaviours, orientations and skewness. Many of the attributes of the bivariate stable distribution are controlled by a measure, which we model using a flexible Bernstein polynomial basis prior. The method is the first attempt to incorporate non-Gaussian kernels in a two-dimensional integro-difference equation model and will be shown to improve prediction over the Gaussian kernel model for a data set of Pacific sea surface temperatures.

We present a Bayesian nonparametric model for ordinal responses, which is based on mixture modelling for the joint distribution of covariates and latent continuous responses. The modelling framework enables flexible inference for both the regression relationships and other aspects of the conditional response distribution. In typical ordinal regression models, assumptions include linearity and parametric distributions. These assumptions may be restrictive for many applications, and computational challenges arise when fitting these models. The nonparametric ordinal regression model avoids these challenges. This modelling approach falls into the class of Bayesian nonparametric density regression methods, which we also review. The density regression modelling approach is illustrated with both continuous and ordinal responses through an application to estimate the relationships between ozone concentration and other environmental characteristics.

ABSTRACT We propose a flexible approach to modeling for renewal processes. The model is built from a structured mixture of Erlang densities for the renewal process inter-arrival density. The Erlang mixture components have a common scale parameter, and the mixture weights are defined through an underlying distribution function modeled nonparametrically with a Dirichlet process (DP) prior. This model specification enables nonstandard shapes for the inter-arrival time density, including heavy tailed and multimodal densities. Moreover, the choice of the DP centering distribution controls clustering or declustering patterns for the point process, which can therefore be encouraged in the prior specification. Using the analytically available Laplace transforms of the relevant functions, we study the renewal function and the directly related K function, which can be used to infer about clustering or declustering patterns. From a computational point of view, the model structure is attractive as it enables efficient posterior simulation while properly accounting for the likelihood normalizing constant implied by the renewal process. A hierarchical extension of the model allows for the quantification of the impact of different levels of a factor. The modeling approach is illustrated with several synthetic datasets, earthquake occurrences data, and coal-mining disaster data.

We develop two prior distributions for probability vectors which, in contrast to the popular Dirichlet distribution, retain sparsity properties in the presence of data. Our models are appropriate for count data with many categories, most of which are expected to have negligible probability. Both models are tractable, allowing for efficient posterior sampling and marginalization. Consequently, they can replace the Dirichlet prior in hierarchical models without sacrificing convenient Gibbs sampling schemes. We derive both models and demonstrate their properties. We then illustrate their use for model-based selection with a hierarchical model in which we infer the active lag from time-series data. Using a squared-error loss, we demonstrate the utility of the models for data simulated from a nearly deterministic dynamical system. We also apply the prior models to an ecological time series of Chinook salmon abundance, demonstrating their ability to extract insights into the lag dependence.

Modeling and inference for survival analysis problems typically revolves around different functions related to the survival distribution. Here, we focus on the mean residual life (MRL) function, which provides the expected remaining lifetime given that a subject has survived (i.e. is event-free) up to a particular time. This function is of direct interest in reliability, medical, and actuarial fields. In addition to its practical interpretation, the MRL function characterizes the survival distribution. We develop general Bayesian nonparametric inference for MRL functions built from a Dirichlet process mixture model for the associated survival distribution. The resulting model for the MRL function admits a representation as a mixture of the kernel MRL functions with time-dependent mixture weights. This model structure allows for a wide range of shapes for the MRL function. Particular emphasis is placed on the selection of the mixture kernel, taken to be a gamma distribution, to obtain desirable properties for the MRL function arising from the mixture model. The inference method is illustrated with a data set of two experimental groups and a data set involving right censoring. The supplementary material available at Biostatistics online provides further results on empirical performance of the model, using simulated data examples.