Matrix-variate data with missing entries arise frequently in applications where observations are naturally organized as two-dimensional arrays. Although the matrix normal distribution provides a parsimonious model through its Kronecker covariance structure, standard EM estimation can be computationally expensive because arbitrary missingness patterns typically destroy this separability in the E-step. In this paper, we propose an efficient partial EM algorithm for matrix-variate normal data with missing entries. The proposed method updates the conditional mean and covariance of the missing component through coordinate-wise approximations, avoiding repeated inversion of pattern-specific covariance matrices and avoiding construction of the full vectorized covariance matrix. We further develop a specialized update for submatrix missingness, where the missing-block precision retains a Kronecker product structure, and the covariance update can be carried out independently in the row and column directions. Simulation studies show that the proposed methods substantially reduce computation time compared with exact EM while preserving nearly identical observed-data likelihood across a range of dimensions and missing proportions. A real-data application to hyperspectral image patches demonstrates that the proposed imputation strategy can be embedded within a matrix-variate mixture model for simultaneous imputation and clustering.

The stochastic EM algorithm replaces the E-stepwith a Monte Carlo approximation, trading monotonicity for the potential to escape local maxima. Estimation techniques include averaging the tail of the chain, and choosing the value in the chain associated with the largest likelihood value. We demonstrate that the latter estimator diverges from the maximum likelihood estimate with high probability as the dimensionality of the parameter increases, but that it is also more precise in terms of chain length when the parameter is a scalar. Based on these findings, we propose a new estimator which we prove achieves this same level of precision for inference of multidimensional parameters. Simulation studies demonstrate the benefits of the proposed estimator when compared to topical approaches. MSC2020 subject classifications: Primary 62F12, 62F10.

We propose a new model and approach for assessing count measurement systems when a gold standard or the true count is available. We assume that the true counts arise from a Poisson distribution and the observed counts are subject to two types of errors. First, we may miss a true defect and second, we may obtain a false positive by labeling something flawless as a defect. We initially consider a gold standard plan where we have access to all the information. Then we consider a second plan where we only know the true count. We compare and discuss these plans empirically and theoretically.

Introduction Cluster analysis, a machine learning-based and data-driven technique for identifying groups in data, has demonstrated its potential in a wide range of contexts. However, critical appraisal and reproducibility are often limited by insufficient reporting, ultimately hampering the interpretation and trust of key stakeholders. The present paper describes the protocol that will guide the development of a reporting guideline and checklist for studies incorporating cluster analyses—Transparent Reporting of Cluster Analyses.Methods and analysis Following the recommended steps for developing reporting guidelines outlined by the Enhancing the QUAlity and Transparency Of health Research Network, the work will be divided into six stages. Stage 1: literature review to guide development of initial checklist. Stage 2: drafting of the initial checklist. Stage 3: internal revision of checklist. Stage 4: Delphi study in a global sample of researchers from varying fields (n=≈) to derive consensus regarding items in the checklist and piloting of the checklist. Stage 5: consensus meeting to consolidate checklist. Stage 6: production of statement paper and explanation and elaboration paper. Stage 7: dissemination via journals, conferences, social media and a dedicated web platform.Ethics and dissemination Due to local regulations, the planned study is exempt from the requirement of ethical review. The findings will be disseminated through peer-reviewed publications. The checklist with explanations will also be made available freely on a dedicated web platform (troca-statement.org) and in a repository.

The logistic normal multinomial distribution is gaining interest in modelling microbiome data. It utilizes a hierarchical structure such that the observed counts conditional on the compositions are assumed to be multinomial random variables and the log-ratio transformed compositions are assumed to be from a Gaussian distribution. While multinomial distribution accounts for the compositional nature of the data, and a Gaussian prior offers flexibility in the structure of covariance matrices, the log-ratio transformed compositions of the microbiome data can be highly skewed, especially at a lower taxonomic level. Thus, a Gaussian distribution may not be an ideal prior for the log-ratio transformed compositions. A novel mixture of logistic skew-normal multinomial (LSNM) distribution is proposed in which a multivariate skew-normal distribution is utilized as a prior for the log-ratio transformed compositions. A variational Gaussian approximation in conjunction with the EM algorithm is utilized for parameter estimation.

We introduce a parameterization for the multivariate skew normal and skew- t distributions, which enforces an orthogonal structure on the skewness parameter. This approach provides substantial benefits in computational efficiency during parameter estimation, resulting in a model which strikes an excellent balance between flexibility and model-fitting feasibility. We illustrate this primarily through implementing the proposed distributions in a mixture model-based clustering framework. We compare to competing skew distributions via both simulated and real data analyses, reporting both computation time and model-fit metrics.

Sketching algorithms use random projections to generate a smaller sketched data set, often for the purposes of modelling. Complete and partial sketch regression estimates can be constructed using information from only the sketched data set or a combination of the full and sketched data sets. Previous work has obtained the distribution of these estimators under repeated sketching, along with the first two moments for both estimators. Using a different approach, we also derive the distribution of the complete sketch estimator, but additionally consider the error term under both repeated sketching and sampling. Importantly, we obtain pivotal quantities which are based solely on the sketched data set which specifically not requiring information from the full data model fit. These pivotal quantities can be used for inference on the full data set regression estimates or the model parameters. For partial sketching, we derive pivotal quantities for a marginal test and an approximate distribution for the partial sketch under repeated sketching or repeated sampling, again avoiding reliance on a full data model fit. We extend these results to include the Hadamard and Clarkson-Woodruff sketches then compare them in a simulation study.

When modeling the functional relationship between a response variable and covariates via linear regression, multiple relationships may be present depending on the underlying component structure. Deploying a flexible mixture distribution can help with capturing a wide variety of such structures, thereby successfully modeling the response–covariate relationship while addressing the components. In that spirit, a mixture regression model based on the finite mixture of generalized hyperbolic distributions is introduced, and its parameter estimation method is presented. The flexibility of the generalized hyperbolic distribution can identify better-fitting components, which can lead to a more meaningful functional relationship between the response variable and the covariates. In addition, we introduce an iterative component combining procedure to aid the interpretability of the model. The results from simulated and real data analyses indicate that our method offers a distinctive edge over some of the existing methods, and that it can generate useful insights on the data set at hand for further investigation.

We propose a variant of the bootstrap to assess the variability of posterior probabilities arising from Gaussian model-based clustering. The bootstrap variant uses predictions based on out-of-bootstrap-sample observations and then constructs confidence intervals for the posterior probabilities using the percentile method. The methodology outperforms the multivariate Delta method approach when comparing empirical coverage probabilities on simulated data. The proposed and multivariate Delta methods are also illustrated on the well-known Iris data set.

Bi-clustering is a technique that allows for the simultaneous clustering of observations and features in a dataset. This technique is often used in bioinformatics, text mining, and time series analysis. An important advantage of biclustering algorithm is the ability to uncover multiple ``views'' (i.e., through rows and column groupings) in the data. Several Gaussian mixture model based biclustering approach currently exist in the literature. However, they impose severe restrictions on the structure of the covariance matrix. Here, we propose a Gaussian mixture model-based bi-clustering approach that provides a more flexible block-diagonal covariance structure. We show that the clustering accuracy of the proposed model is comparable to other known techniques but our approach provides a more flexible covariance structure and has substantially lower computational time. We demonstrate the application of the proposed model in bioinformatics and topic modelling.

Generalized linear models are a popular analytics tool with interpretable results and broad applicability, but require iterative estimation procedures that impose data transfer and computational costs that can be problematic under some infrastructure constraints. We propose a doubly-sketched approximation of the iteratively re-weighted least squares algorithm to estimate generalized linear model parameters using a sequence of surrogate datasets. The procedure sketches once to reduce data transfer costs, and sketches again to reduce data computation costs, yielding wall-clock time savings. Regression coefficients and standard errors are produced, with comparison against literature methods. Asymptotic properties of the proposed procedure are shown, with empirical results from simulated and real-world datasets. The efficacy of the proposed method is investigated across a variety of commodity computational infrastructure configurations accessible to practitioners. A highlight of the present work is the estimation of a Poisson-log generalized linear model across almost 1.7 billion observations on a personal computer in 25 min.

Families of mixtures of multivariate power exponential (MPE) distributions have already been introduced and shown to be competitive for cluster analysis in comparison to other mixtures of elliptical distributions, including mixtures of Gaussian distributions. A family of mixtures of multivariate skewed power exponential distributions is proposed that combines the flexibility of the MPE distribution with the ability to model skewness. These mixtures are more robust to variations from normality and can account for skewness, varying tail weight, and peakedness of data. A generalized expectation-maximization approach, which combines minorization-maximization and optimization based on accelerated line search algorithms on the Stiefel manifold, is used for parameter estimation. These mixtures are implemented both in the unsupervised and semi-supervised classification frameworks. Both simulated and real data are used for illustration and comparison to other mixture families.

A dataset may exhibit multiple class labels for each observation; sometimes, these class labels manifest in a hierarchical structure. A textbook analogy would be that a book can be labelled as statistics as well as the encompassing label of non-fiction. To capture this behaviour in a model-based clustering context, we describe a model formulation and estimation procedure for performing clustering with nested Gaussian clusters in orthogonal intrinsic variable subspaces. We elucidate a two-stage clustering model, whereby the observed manifest variables are assumed to be a rotation of intrinsic primary and secondary clustering subspaces with additional noise subspaces. In a hierarchical sense, secondary clusters are presumed to be subclusters of primary clusters and so share Gaussian cluster parameters in the primary cluster subspace. An estimation procedure using the expectation-maximization algorithm is provided, with model selection via Bayesian information criterion. Real-world datasets are evaluated under the proposed model.

We present a parsimonious dual-subspace clustering approach for a mixture of matrix-normal distributions. By assuming certain principal components of the row and column covariance matrices are equally important, we express the model in fewer parameters without sacrificing discriminatory information. We derive update rules for an ECM algorithm and set forth necessary conditions to ensure identifiability. We use simulation to demonstrate parameter recovery, and we illustrate the parsimony and competitive performance of the model through two data analyses.

Mixtures of multivariate leptokurtic-normal distributions have been recently introduced in the clustering literature based on mixtures of elliptical heavy-tailed distributions. They have the advantage of having parameters directly related to the moments of practical interest. We derive two estimation procedures for these mixtures. The first one is based on the majorization-minimization algorithm, while the second is based on a fixed point approximation. Moreover, we introduce parsimonious forms of the considered mixtures and we use the illustrated estimation procedures to fit them. We use simulated and real data sets to investigate various aspects of the proposed models and algorithms.

We extend the literature regarding assessments of agreement between soft/fuzzy/probabilistic cluster allocations by providing closed-form approaches for two measures which behave as fuzzy generalizations of the popular adjusted Rand index (ARI): one novel and one previously requiring a Monte Carlo estimation process. Both of these measures retain the reflexive property of the ARI—an arguably essential property for the interpretability of a cluster agreement measure—and both are feasible in their closed-form for sample sizes ranging into five digits or more using standard consumer computers. We describe the approximate computational complexity in each case, and apply both measures in simulated and real data contexts.

The expectation-maximization (EM) algorithm for incomplete data with highly diverse missing data patterns can be computationally expensive. A partial expectation-maximization (PEM) algorithm is developed to ease this computational burden. This PEM algorithm circumvents the need for a traditional E-step by performing a partial E-step that reduces the Kullback-Leibler divergence between the conditional distribution of the missing data and the distribution of the missing data given the observed data. The PEM and EM algorithms are compared in terms of computation time and convergence on simulated data. The PEM algorithm is illustrated using a latent Gaussian mixture model to cluster a white bread sensory analysis dataset.

A major challenge when performing model-based clustering is a large increase in the number of free parameters as the data dimensionality increases. To combat this issue, parsimonious methods such allow component covariance matrices to share parameters by exploiting geometric redundancies. The present work considers an additional level of intracluster structure that also captures hybridisation of mean and covariance parameters between components for the multivariate normal distribution. We posit components with heterogeneous parameterisation; a subset are considered factor components and have explicit mean and covariance parameters, and the remainder are considered hybrid components that have means and covariances implied by a set of factor loadings that weight factor component parameters. An estimation procedure is provided using the Expectation-Maximization algorithm, and comparison to Gaussian mixture models with parsimonious covariances is made by evaluation on a collection of datasets.

We introduce a model-based approach for clustering multivariate functional data observations. We utilize theoretical results regarding a surrogate density on the truncated Karhunen–Loeve expansions along with a direct sum specification of the functional space to define a matrix normal distribution on functional principal components. This formulation allows for individual parsimonious modelling of the function space and coefficient space of the univariate components of the multivariate functional observations in the form a subspace projection and latent factor analyzers, respectively. The approach facilitates interpretation at both the full multivariate level and the component level, which is of specific interest when the component functions have clear meaning. We derive an AECM algorithm for fitting the model, and discuss appropriate initialization strategies, convergence and model selection criteria. We demonstrate the model’s applicability through simulation and two data analyses on observations that have many functional components.