Despite the significant consequences of delisting, there have been only a few studies of business failure prediction that concentrate on delisting, particularly for the U.S. market. This study aims to predict delisting using quarterly financial ratios of 8,870 companies ever listed in the major U.S. exchanges from 1970 to 2022, along with key economic indices. We construct our data set to build a delisting prediction model that is robust across different company phases and economic conditions. To enhance predictive performance, we build an ensemble model to predict delisting, integrating five widely used machine learning methods : Logistic Regression, Random Forest, Gradient Boosting, Support Vector Machine, and Neural Network. We compare the predictive performance of the five base learners and the ensemble model. The ensemble model achieves an accuracy of 0.836 and MCC of 0.457, demonstrating comparable accuracy and an improved MCC relative to the top-performing base learner, Random Forest. We leverage MCC, a reliable measure for imbalanced response data, to determine a classification threshold and comprehensively evaluate the prediction results. We find that the Price-Earnings ratio, profitability ratios and inflation are among the most informative predictors for forecasting, aligning with prior research.

Student attrition is an important issue for higher education as it brings about grave costs to both students and institutions. This article investigates the 2-year persistence rate of students enrolled at a large 4-year public institution in California in Fall 2016 to Fall 2020. Predictors considered include student demographic information, socioeconomic variables, academic preparation, and their academic performance at the institution. Two analytical approaches are used, discrete-time survival analysis and random forest (RF). The results indicate that academic performance variables after enrollment are most strongly associated with 2-year persistence. In particular, monitoring and providing help promptly to students with earned-units below 6 or GPA below 2.0 in the first term may prevent them from dropping out. Among socioeconomic variables available for analysis, first-generation status seems to be more directly related to student attrition. It is also illustrated how the RF model may be used to provide individualized prediction of 2-year persistence.

Greedy search (GS) or exhaustive search plays a crucial role in decision trees and their various extensions. We introduce an alternative splitting method called smooth sigmoid surrogate (SSS) in which the indicator threshold function used in GS is approximated by a smooth sigmoid function. This approach allows for parametric smoothing or regularization of the erratic and discrete GS process, making it more effective in identifying the true cutoff point, particularly in the presence of weak signals, as well as less prone to the inherent end-cut preference problem. Additionally, SSS provides a convenient means of evaluating the best split by referencing a parametric nonlinear model. Moreover, in many variants of recursive partitioning, SSS can be reformulated as a one-dimensional smooth optimization problem, rendering it computationally more efficient than GS. Extensive simulation studies and real data examples are provided to evaluate and demonstrate its effectiveness.

The federal funds target rate is commonly considered to be an important indicator of the state of the US economy and is of keen interest to individual investors, financial firms, and other economic agents. In this paper, we focus on the discrete changes in the federal funds target rate during the period from January 1994 to June 2022 and apply the ordinal forest model, a random forest-based prediction method for ordinal response variable. We examine the model's performance with 45 predictor variables which include macroeconomic and financial variables as well as forward-looking survey measures. For an accurate and honest measure of the model performance, we employ single-period-ahead out-of-sample forecasting accuracy instead of evaluating the in-sample fit. Our empirical results show the ordinal forest method significantly outperforms a benchmark that uses the most recent data among previous studies on federal funds target rate. We find that TB spread is the most informative from a forecasting perspective along with GDP, initial jobless claims, and survey measures.

Traditional methods for evaluating a potential treatment have focused on the average treatment effect. However, there exist situations where individuals can experience significantly heterogeneous responses to a treatment. In these situations, one needs to account for the differences among individuals when estimating the treatment effect. Li et al. (2022) proposed a method based on random forest of interaction trees (RFIT) for a binary or categorical treatment variable, while incorporating the propensity score in the construction of random forest. Motivated by the need to evaluate the effect of tutoring sessions at a Math and Stat Learning Center (MSLC), we extend their approach to an ordinal treatment variable. Our approach improves upon RFIT for multiple treatments by incorporating the ordered structure of the treatment variable into the tree growing process. To illustrate the effectiveness of our proposed method, we conduct simulation studies where the results show that our proposed method has a lower mean squared error and higher optimal treatment classification, and is able to identify the most important variables that impact the treatment effect. We then apply the proposed method to estimate how the number of visits to the MSLC impacts an individual student’s probability of passing an introductory statistics course. Our results show that every student is recommended to go to the MSLC at least once and some can drastically improve their chance of passing the course by going the optimal number of times suggested by our analysis.

With the collection and availability of data on student academic performance and academic background, higher education institutions have recently stepped up initiatives in and infrastructure for learning analytics, leveraging this deluge of data to inform student success. With definitions of student success varying from analyses of what predicts levels of specific career readiness competencies to degree completion, the environment is a fertile ground for statistical practice and collaboration among a statistically savvy yet diverse clientele of instructors, programme advisors and administrators. In this paper, we discuss our experiences to this end through a consulting project evaluating the impact of writing course class size on students achieving a graduation writing requirement. In detailing the workflow for and challenges in this project, we share aspects of statistical communication and reporting, applications of innovative statistical methodology developed by our research group for handling confounding factors and correlated inputs and training through an interdisciplinary applied institutional research professional development programme. This paper illustrates how instilling an appreciation for statistical inference through each of these components is invaluable for capturing institutional buy-in for data-informed decision-making in general statistical practice.

Functional connectivity MRI (fcMRI) is a technique used to study the functional connectedness of distinct regions of the brain by measuring the temporal correlation between their blood oxygen level-dependent (BOLD) signals. fcMRI is typically measured with the Pearson correlation (PC), which assumes that there is no lag between time series. Dynamic time warping (DTW) is an alternative measure of similarity between time series that is robust to such time lags. We used PC fcMRI data and DTW fcMRI data as predictors in machine learning models for classifying autism spectrum disorder (ASD). When combined with dimension reduction techniques, such as principal component analysis, functional connectivity estimated with DTW showed greater predictive ability than functional connectivity estimated with PC. Our results suggest that DTW fcMRI can be a suitable alternative measure that may be characterizing fcMRI in a different, but complementary, way to PC fcMRI that is worth continued investigation. In studying different variants of cross validation (CV), our results suggest that, when it is necessary to tune model hyperparameters and assess model performance at the same time, a K-fold CV nested within leave-one-out CV may be a competitive contender in terms of performance and computational speed, especially when sample size is not large.

Background and Objectives Selecting applications for college admission is critical for university operation and development. This paper leverages machine learning techniques to support enrollment management teams through data-informed decision-making in this otherwise laborious admissions processing. Research Design and Measures Two aspects of university admissions are considered. An ensemble learning approach, through the SuperLearner algorithm, is used to predict student show (yield) rate. The goal is to improve prediction accuracy to minimize over- or under-enrollment. A combinatorial optimization framework is proposed to weigh academic performance and experiential factors for ranking and selecting students for admission. This framework uses simulated annealing, and an efficacy study is presented to evaluate performance. Results The proposed framework is illustrated for selecting an incoming class by optimizing predicted graduation rate and by developing an eligibility index. Each example presents a selection process under potential academic performance and experiential factor targets a university may place on an admitted class. R code is provided for higher education researchers and practitioners to apply the proposed methods in their own settings.

Individuals may respond to treatments with significant heterogeneity. To optimize the treatment effect, it is necessary to recommend treatments based on individual characteristics. Existing methods in the literature for learning individualized treatment regimes are usually designed for randomized studies with binary treatments. In this study, we propose an algorithm to extend random forest of interaction trees (Su et al., 2009) to accommodate multiple treatments. By integrating the generalized propensity score into the interaction tree growing process, the proposed method can handle both randomized and observational study data with multiple treatments. The performance of the proposed method, relative to existing approaches in the literature, is evaluated through simulation studies. The proposed method is applied to an assessment of multiple voluntary educational programmes at a large public university.

This chapter provides an additional ensemble learning method that combines predictions from eight popular machine learning methods. The observational study, as an alternative to the randomized experiment, can not only avoid the possible moral hazard but also be less expensive. A well-designed randomized experiment can yield unbiased treatment effect because the randomized treatment allocation balances the baseline covariates between treated and control subjects. However, a randomized trial is always feasible due to ethical, practical reasons. Logistic regression (LR) is typically used to estimate propensity scores where treatment status is regressed on a set of observed covariates. LR is a strong tool for statistical analysis; however, as McCaffrey et al. points out, large numbers of covariates tend to hurt its ability to accurately estimate propensity scores, as a result of multi-collinearity. The final matched set will have an equal number of treated to control subjects, with the goal of having a balanced distribution for each covariate between the two treatment groups.

Propensity score matching and weighting methods are applied to balance covariates and reduce selection bias in the analysis of observational study data, and ultimately estimate a treatment effect. We wish to evaluate the impact of a Supplemental Instruction (SI) program on student success in an Introductory Statistics course. In such student success studies, propensity score methods have been applied successfully for evaluating a binary treatment, namely whether a student attending SI performs better or worse than a student who does not attend SI. However, in this setting, we also want to draw inferences on the dose-response relationship, namely how does the number of times a student attends SI impact performance in the course. In this paper, we introduce generalized propensity scores (GPS) for analyzing such continuous treatment. We extend recent developments in GPS analyses from the personalized learning literature for evaluating SI engagement on student success. As part of the exposition, we provide a brief review of generalized propensity scores, compare our proposed GPS approaches, present guidelines on how these methods can be applied to educational data, and present R code and illustration for practitioners to use as a template in educational data mining applications.

Clinical unmet needs still abound for an accurate noninvasive test to facilitate early detection of lung cancer. Here, we describe a novel combination of clinical, imaging and DNA methylation biomarkers to improve the classification of pulmonary nodules. We conducted a prospective collection and retrospective blinded evaluation trial comprising 1,380 subjects in 24 sites. We developed a 10-feature combined clinical and imaging biomarkers model (CIBM) for the classification of malignant and benign pulmonary nodules in a cohort (n=839) and validated it in 2 cohorts (n1=258, n2=283). Then we integrated CIBM model with our previously established ctDNA methylation model (PulmoSeek) to create a new combined model (n=258), PulmoSeek Plus, and verified it independently (n=283). Meanwhile, a 12-feature imaging biomarker model for invasiveness differentiation (IBMI) of lung adenocarcinoma were established (n=624) and validated (n1=202, n2=193). Clinical utility of the models was evaluated using a decision curve analysis. The CIBM model achieved improved AUCs (0.85 [95% CI 0.80-0.89]; 0.85 [0.81-0.89]) over Mayo model (0.60 [0.52-0.68]; 0.57 [0.50-0.64]) and Brock model (0.70 [0.63-0.77]; 0.67 [0.60-0.73]) in the two validation cohorts, respectively. PulmoSeek Plus had AUC of 0.90 [0.88-0.93] in the combined set (n=541), significantly outperforming both CIBM (0.85 [0.82-0.88]) and PulmoSeek (0.85 [0.82-0.88]). The overall sensitivity was 98.0% [0.97-1.00] at a fixed specificity of 50.0% for rule out. High sensitivity of 98.0% [0.96-0.99] was maintained in early-stage lung cancer (0-I, n=390) and 99.2% [0.96-1.00] in 5-10 mm nodules (n=123). The IBMI model had AUCs of 0.87 [0.82-0.92], 0.89 [0.84-0.93] and 0.89 [0.85-0.92] in the two validation sets and the combined set. At a risk score of 0.54, PulmoSeek Plus improved net benefit by 60.8%, equivalent to detecting additional 82.4% of lung cancers. Using two cut-offs of PulmoSeek Plus scores to reclassify would have reduced 46.0% unnecessary surgeries and reduced 73.2% delayed treatment. PulmoSeek Plus model improves the early detection and classification of pulmonary nodules, potentially worth using in clinical decision-making.

Observational studies require matching across groups over multiple confounding variables. Across the literature, matching algorithms fail to handle the issue of missing data. Consequently, missing values are regularly imputed prior to being considered in the matching process. However, imputing is not always practical, forcing us to drop an observation due to the deficiency of the chosen algorithm, decreasing the power of the study and possibly failing to capture crucial latent information. We propose a missing data mechanism to incorporate within an iterative multivariate matching method. The underlying framework utilizes random forest as a natural tool in constructing a distance matrix, implemented with surrogate splits where there might be missing values. The output is then easily fed into an optimal matching algorithm. We apply this method to evaluate the effectiveness of supplemental instruction (SI) sessions, a voluntary program where students seek additional help, in a large enrollment, bottleneck introductory business statistics course. This is an observational study with two groups, those who attend multiple SI sessions and those who do not, and, as typical in educational data mining, challenged by missing data. Additionally, we perform a data simulation on missingness to further demonstrate the efficacy of our proposed approach.

The null distribution of the likelihood ratio test (LRT) of a onecomponent normal model versus two-component normal mixture model is unknown.In this paper, we take a bootstrap approach to the likelihood ratio test for testing bimodality of plasma glucose concentrations from Rancho Bernardo Diabetes Study.The small p-values from this approach support the hypothesis that a bimodal normal mixture model fits the data significantly better than a unimodal normal model.The size and power of the bootstrap based LRT are evaluated through simulations.The results suggest that a sample size of close to 500 would be necessary in order to attain a power of 90% for detecting the unbalanced mixtures with means and variances similar to those in the Rancho Bernardo data.Besides sample size, the power also depends on the two means and variances of the two components in the data.

We expand methods for estimating an optimal treatment regime (OTR) from the personalized medicine literature to educational data mining applications. As part of this development, we detail and modify the current state-of-the-art, assess the efficacy of the approaches for student success studies, and provide practitioners the machinery to apply the methods in their specific problems. Our particular interest is to estimate an optimal treatment regime for students enrolled in an introductory statistics course at San Diego State University (SDSU). The available treatments are combinations of three programs SDSU implemented to foster student success in this large enrollment, bottleneck STEM course. We leverage tree-based reinforcement learning approaches based on either an inverse probability-weighted purity measure or an augmented probability-weighted purity measure. The thereby deduced OTR promises to significantly increase the average grade in the introductory course and also reveals the need for program recommendations to students as only very few, on their own, selected their optimal treatment.

AbstractIn this article, we present the random forest algorithm by Breiman in 2001, discuss a few of the most popular random forest extensions, and present random forest of interaction trees for both randomized trials and observational data. In particular, extremely randomized trees, acceptance‐rejection trees, and conditional random forest are detailed and compared in terms of algorithm differences, prediction accuracy for classification and regression problems, computation time, and variable selection bias. We also briefly highlight additional random forest‐related methods and algorithms, while providing references of relevant articles in the literature and, if available, R packages. Random forest of interaction trees (RFIT) is described within the context of precision medicine for randomized clinical trials as well as RFIT incorporating propensity score appropriate for data from observational studies. A simulation study demonstrates that this RFIT approach can improve prediction accuracy of the individualized treatment effect (ITE) for observational study data. The RFIT method is applied to a study of headache treatments, providing individualized treatment effect for all patients, variable importance rankings, and the effects of the most important variables on the ITE via partial dependence plots.

Propensity score methods account for selection bias in observational studies. However, the consistency of the propensity score estimators strongly depends on a correct specification of the propensity score model. Logistic regression and, with increasing popularity, machine learning tools are used to estimate propensity scores. We introduce a stacked generalization ensemble learning approach to improve propensity score estimation by fitting a meta learner on the predictions of a suitable set of diverse base learners. We perform a comprehensive Monte Carlo simulation study, implementing a broad range of scenarios that mimic characteristics of typical data sets in educational studies. The population average treatment effect is estimated using the propensity score in Inverse Probability of Treatment Weighting. Our proposed stacked ensembles, especially using gradient boosting machines as a meta learner trained on a set of 12 base learner predictions, led to superior reduction of bias compared to the current state-of-the-art in propensity score estimation. Further, our simulations imply that commonly used balance measures (averaged standardized absolute mean differences) might be misleading as propensity score model selection criteria. We apply our proposed model - which we call GBM-Stack - to assess the population average treatment effect of a Supplemental Instruction (SI) program in an introductory psychology (PSY 101) course at San Diego State University. Our analysis provides evidence that moving the whole population to SI attendance would on average lead to 1.69 times higher odds to pass the PSY 101 class compared to not offering SI, with a 95% bootstrap confidence interval of (1.31, 2.20).

In this article, we present the random forest algorithm by Breiman in 2001, discuss a few of the most popular random forest extensions, and present random forest of interaction trees for both randomized trials and observational data. In particular, extremely randomized trees, acceptance-rejection trees, and conditional random forest are detailed and compared in terms of algorithm differences, prediction accuracy for classification and regression problems, computation time, and variable selection bias. We also briefly highlight additional random forest-related methods and algorithms, while providing references of relevant articles in the literature and, if available, R packages. Random forest of interaction trees (RFIT) is described within the context of precision medicine for randomized clinical trials as well as RFIT incorporating propensity score appropriate for data from observational studies. A simulation study demonstrates that this RFIT approach can improve prediction accuracy of the individualized treatment effect (ITE) for observational study data. The RFIT method is applied to a study of headache treatments, providing individualized treatment effect for all patients, variable importance rankings, and the effects of the most important variables on the ITE via partial dependence plots.

Nocturnal hypoglycemia is a common phenomenon among patients with diabetes and can lead to a broad range of adverse events and complications. Identifying factors associated with hypoglycemia can improve glucose control and patient care. We propose a repeated measures random forest (RMRF) algorithm that can handle nonlinear relationships and interactions and the correlated responses from patients evaluated over several nights. Simulation results show that our proposed algorithm captures the informative variable more often than naïvely assuming independence. RMRF also outperforms standard random forest and extremely randomized trees algorithms. We demonstrate scenarios where RMRF attains greater prediction accuracy than generalized linear models. We apply the RMRF algorithm to analyze a diabetes study with 2524 nights from 127 patients with type 1 diabetes. We find that nocturnal hypoglycemia is associated with HbA1c, bedtime blood glucose (BG), insulin on board, time system activated, exercise intensity, and daytime hypoglycemia. The RMRF can accurately classify nights at high risk of nocturnal hypoglycemia.