Abstract Trust is a basic necessity for translating machine learning models from bench to bedside. To foster clinicians’ trust in predictive models, these models should be interpretable or explainable. A major prerequisite for interpretability or explainability is that feature subsets are stable, i.e., that predictors returned by feature or best subset selection approaches do not change dramatically when the data is perturbed only slightly. However, prior research has shown that stable selections are usually not possible when the number of labels in the smaller class per feature, or outcome events per variable (EPV), is very low, and it remains challenging to build stable, interpretable models in such cases. In this work, we conduct a narrative review in the very low EPV regime and propose to combine cardinality-constrained best subset selection for logistic regression with resampling to tackle the selection instability. We perform a comprehensive evaluation on both synthetic and real-world clinical data, and find that our novel resampling-based model selection procedure yields interpretable feature subsets with a better discriminative performance and stability compared to best subset selection without resampling as well as feature selection with and without resampling. A Python package implementing the proposed methods is available at https://github.com/ml-lab-htw/l0l2learn .
Abstract Acute diarrheal disease is a leading cause of death in children under age 5, disproportionately impacting children in low-resource settings. Although many cases could respond to antibiotic treatment, the benefits of widely prescribing antibiotics must be weighed against the risks of antimicrobial resistance. These challenges motivate development of individualized treatment guidelines for diarrheal disease. In this study, we utilize a framework for creation and evaluation of individualized treatment rules that leverage diagnostic and clinical information to make treatment recommendations. In contrast to many applications of pipelines for creating and evaluating treatment rules, we (i) explicitly create rules that limit the proportion of children treated based on expected clinical benefit, to limit risks for overtreatment and the emergence of antimicrobial resistance, and (ii) propose methods to compare rules based on different sets of input covariates, allowing for quantification of the impact of measuring additional biomarkers in clinical settings. We use a nested cross-validation procedure with ensemble machine learning and doubly-robust estimation to derive, evaluate, and compare rules. We demonstrate that our proposed method yields appropriate inference in a realistic simulation study and apply our method to data from the AntiBiotics for Children with severe Diarrhea (ABCD) trial.
Studies of HPV vaccine efficacy usually record infections with vaccine targeted and non-targeted strains. Contrary to blinded randomized controlled trials, confounding bias can be a threat and risk compensation may occur in observational studies. Etievant et al. (Biometrics, 2023) proposed to use cervical infections with non-targeted HPV strains to remove or reduce confounding bias in estimates of vaccine efficacy on targeted strains. However, they assumed that vaccinated women could not change their behavior after vaccination. This work investigates if the quantity estimated in practice with the method of Etievant et al. has a clear causal meaning under a more plausible setting where unmeasured sexual behavior acts as both a confounder and a mediator. Under certain assumptions, using non-targeted HPV infections can remove both confounding bias and the portion of the vaccine effect on the targeted HPV strains that is mediated through the change of behavior. In that case, the estimated quantity has a clear causal interpretation as it represents the direct immunological effect of the vaccine. Infections with non-targeted HPV strains could also be used to isolate the indirect behavioral effect of the vaccine in unblinded randomized controlled trial.
Targeted Maximum Likelihood Estimation, also referred to as Targeted Minimum Loss Estimation (TMLE), is a statistical method that enables causal inference while incorporating flexible, advanced machine learning techniques. Within the drug development space, both drug sponsors and FDA have leveraged TMLE across a range of relevant clinical settings intended for a variety of regulatory activities. The goal of this article is to provide a holistic overview of TMLE use cases as they have been applied in submissions to the Food and Drug Administration (FDA) Center for Drug Evaluation and Research (CDER) and in CDER's regulatory research activities. First, we provide an overview of regulatory submissions in which TMLE has been considered to support regulatory decision-making. Second, we summarize FDA CDER's regulatory science initiatives that advance analytic capabilities and support the use of TMLE.
Functional data analysis (FDA) provides a powerful statistical framework for analyzing complex data, such as curves or functions, over high-dimensional domains. In this paper, we focus on functional predictor regression (scalar-on-function) models applied to the prediction of birth weight using maternal dietary patterns during pregnancy. Specifically, we analyze trajectories of nine components of the Alternative Healthy Eating Index (AHEI) as multivariate functional predictors. We adopt Multivariate Functional Principal Component Analysis (MVFPCA) to obtain low-dimensional, uncorrelated representations of the multivariate functional predictors through their FPC scores. Building on MVFPCA, we develop a novel multivariate functional principal component regression (MVFPCR) model to predict birth weight effectively. Our method accommodates various regression approaches, including linear and quantile regression, depending on the distribution of birth weights. Through simulation studies and the application to a fetal growth study dataset, we demonstrate the effectiveness of our proposed model in functional predictor regression.
