Abstract In causal inference, properly selecting the propensity score (PS) model is an important topic and has been widely investigated in observational studies. However, there are very few studies investigating the model selection issue for causal inference when the exposure is missing at random (MAR). When the exposure variable contains missing data, an intuitive approach is to impute the missing exposure values first and perform a PS analysis on the imputed dataset under MAR assumption. In this article, we discuss how to select both imputation and PS models. We then propose a new criterion, termed the “rank score,” for evaluating the overall performance of both models. Simulation studies show that the full imputation plus the outcome‐related PS models leads to the smallest root mean squared error, and the rank score can help select the best models. An application study is conducted to quantify the causal effect of cardiovascular disease on the mortality of COVID‐19 patients.
Abstract The rapid growth of big data sets has made subsampling an essential tool for data storage and analysis. In many applications, first‐order models are inadequate, as interaction and quadratic effects are often present. Second‐order models provide a concise framework for capturing such higher‐order relationships, yet subsampling strategies tailored to these models remain underexplored. Hence, we propose an efficient subsampling method for second‐order models (ESSO) inspired by orthogonal arrays. Our method is particularly suited for scenarios with measurement constraints, where subsample selection does not require response information. This study examines the optimal subsampling method under the ‐optimality criterion for second‐order models. Additionally, the asymptotic normality of the estimator is established, and a sequential algorithm is proposed for practical implementation. Numerical simulations and a real data example further demonstrate the effectiveness of the proposed approach.
Abstract Motivated by the challenges of heterogeneity and high dimensionality in the analysis of modern data, we investigate continuous regularization methods for learning sparse mixture‐of‐experts generalized linear models (MoE‐GLM). Although there are foundational results about regularized estimators in a broad class of regression models including GLMs, to the best of our knowledge, there is no general theoretical result on consistency in estimation and feature selection of these estimators in MoE‐GLM under ultrahigh‐dimensional settings. We address this gap by studying a general class of regularized estimators for sparse MoE‐GLM. Our results also apply to finite mixtures of regressions, a subclass of MoE‐GLM. The methods are implemented using a modified expectation‐maximization algorithm combined with proximal coordinate descent. We evaluate the empirical performance of the methods via simulations, and demonstrate their practical use with a real data analysis.
Abstract The divisibility conditions implicit in a polynomial hierarchy suggest parameter constraints in regression. With this idea, we establish strong and weak hierarchies for both the lasso and relaxed lasso. Our proposal extends prior work on hierarchical lasso, which was mainly concerned with models of degree 2. The implementation of our methods uses existing quadratic programming techniques, and we apply them to both simulated data and examples from the literature. The empirical results compare favourably with existing techniques in terms of low validation error and model size.
Abstract For regression model selection and estimation, we study a small set of candidate models of maximum likelihood from which all information criteria such as the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) choose their models. We show that asymptotically sufficiently large models in this set are all correct models in that they contain all active variables. Such a concentration of correct models in this small set encourages selecting models from this set. The constrained minimum criterion (CMC) of Tsao is well suited for selecting models from this set. As a by‐product of this study, we obtain selection consistency of the CMC. We also compare CMC with AIC and BIC through a simulation study.
