Abstract Background Cohort selection criteria play a critical role in shaping machine learning (ML) model performance and the equity of clinical outcome predictions across demographic groups. In practice, cohort definitions are often influenced by variable and sometimes inconsistent data processing decisions, which may introduce bias and limit the generalizability of ML models. During the COVID-19 pandemic, rapid cohort construction further increased concerns about transparency and fairness in ML-based analyses. Objective This study aimed to systematically examine how cohort selection and data processing decisions influence ML performance and demographic equity in predicting COVID-19–related in-hospital mortality. Methods Using data from the National COVID Cohort Collaborative (N3C), we evaluated 2 sets of cohorts. Set 1 consisted of 16 cohorts derived from 4 primary data processing decisions, including COVID-19 case identification, inpatient inclusion, diagnosis date selection, and admission timestamp availability. Set 2 expanded this design to 64 cohorts by additionally applying provider ID and location identifier filtering. Model performance was assessed using the area under the receiver operating characteristic curve (AUC) across multiple training-testing cohort combinations. Three ML models—logistic regression, random forest, and gradient boosting—were evaluated using 3 analytical approaches: maximum AUC classification, direct AUC regression, and AUC gap analysis. Performance was further examined across demographic subgroups defined by gender, race, and ethnicity. Results This study analyzed data from the N3C, including patients with a first positive COVID-19 diagnosis between August 1, 2020, and December 31, 2021. Data preprocessing, cohort construction, and model development were completed prior to analysis. Cohort selection decisions had a substantial impact on ML model performance. Admission time inclusion or exclusion emerged as the most influential factor in Set 1 and consistently affected model accuracy across analytical approaches. In Set 2, this decision remained important, while additional criteria, particularly provider ID filtering, also significantly influenced results. The importance of specific decisions varied across models and evaluation strategies. Analyses across demographic subgroups showed that data processing decisions affected predictive performance differently by gender, race, and ethnicity. Conclusions Seemingly minor cohort selection and data processing decisions can meaningfully affect both predictive accuracy and demographic equity in ML-based COVID-19 outcome prediction. These findings highlight the risk of bias introduced by differences in cohort definitions and underscore the need for transparent, standardized, and equity-aware cohort selection practices to support fair and reproducible ML research in health care.
Telehealth offers significant potential to improve healthcare delivery, but its benefits are context-driven. It shows clear advantages in consumer-facing, high-virtualization settings and uncertain impacts in high-stakes, tightly coupled processes like Intensive Care Unit (ICU) care. Prior IS research has examined high-virtualization telehealth, such as mental health. In these settings, patients are active agents, and self-efficacy drives outcomes. Less attention was given in the ICU context, where continuous monitoring, rapid decision-making, and coordination are critical. In this paper, we examine tele-ICU, a telehealth technology designed to support ICU care. We use acute myocardial infarction (AMI) as a context to study a low-virtualizable care process. The AMI care process mostly begins with ICUs and requires rapid monitoring, coordination, and reliance on specialized staff. AMI care contributes significantly to hospital readmissions, targeted under the Medicare Hospital Readmission Reduction Program, underscoring their importance. The shortage of cardiac care staff has increased interest in tele-ICU, which can extend expertise across hospitals. However, the comprehensive role of intensivists in the ICU may also affect the impact of tele-ICUs. These countervailing forces motivate our investigation. Drawing on process virtualization theory and the sociotechnical systems perspective, we develop hypotheses about the direct and interaction effects of tele-ICU, health information interchange, and closed-model cardiac intensivist use on AMI outcomes. Using panel data and a control-function approach, we find no direct effects of tele-ICU. We find strong complementarity between tele-ICU and health information interchange, and separately, between tele-ICU and intensivist staffing. Effects vary by organizational level—hospital, joint-venture, or system—highlighting ICU coordination challenges. Our study contributes theoretically by extending process virtualization theory with a sociotechnical perspective to low-virtualization care processes. It helps guide hospital administrators on when tele-ICU can improve the ICU care process. It contributes policy-wise by informing telehealth investment, integration, and post-pandemic efforts to enhance complex care.
Data analytics has emerged as a crucial tool for understanding the multifaceted impacts of the COVID-19 pandemic. By collecting and analyzing extensive datasets, researchers have gained valuable insights into the virus's transmission, severity, and the effectiveness of public health measures. Yet, many contradictive and non-reproducible results have been published. The significance of cohort representativeness in this context cannot be overstated, as diverse cohorts provide a comprehensive understanding of how various demographic and clinical factors influence COVID-19 outcomes. This study investigates the impact of decision-making processes on cohort diversity, focusing on demographic categories including sex, race, and ethnicity. Results show that decisions made during data preprocessing and cohort construction increase variability in demographic distribution. Specifically, the difference in female representation varied by 0.77% to 2.68%, in Black race from 1.17% to 5.15%, and in Hispanic or Latino ethnicity from 5.84% to 8.21%. It highlights how arbitrary decisions can lead to varying data including changes to data distribution. Seemingly unrelated to demographics factors, including timing and provider selection, significantly influence patient distribution and outcomes, underscoring the necessity for informed, data-driven strategies. The findings emphasize the importance of strategic, evidence-based decision-making to enhance consistency, optimize resource utilization, and effectively serve diverse populations, ultimately contributing to more equitable health outcomes and informed public health policies.
Health Information Technology (HIT) is revolutionizing healthcare by serving as the backbone for various decision support activities across the healthcare continuum, particularly within hospital settings. While existing literature highlights its positive impact on patient satisfaction, costs, and quality, its role in complementing other crucial hospital inputs to influence clinical healthcare outcomes has been relatively understudied. In this study, we explore the complementary effects of a specific type of HIT, Clinical Decision Support Systems (CDSS) on cardiac mortality rates (CMR) in hospitals. Though hospital personnel and cardiac medical services (CMS) are pivotal in reducing CMR, CDSS plays a complementary role by providing information and decision support throughout the cardiac care delivery process. Leveraging panel data spanning from 2016 to 2020, our analysis reveals that CDSS complements CMS and hospital personnel in mitigating CMR. These findings provide theoretical insights into the benefits facilitated by CDSS in cardiac care and hold managerial implications for the effective deployment of this technology within hospital settings. Through our analysis, we aim to elucidate the synergistic effects of CDSS, cardiac medical services, and healthcare personnel in improving clinical healthcare outcomes, particularly in the management of cardiac disease.
The COVID-19 pandemic has had a disproportionate impact on certain racial and ethnic groups, resulting in significant health outcome disparities. The National COVID Cohort Collaborative (N3C) provides a valuable resource for exploring these disparities through big data analytics. This study belongs to a broader work that examines decisions made during data processing and their impact on the analyses performed. Central to our analysis is the introduction of the Continuous Inpatient Encounter (CIE) concept—a novel method we propose for aggregating inpatient visits. By utilizing big data analytics, we aim to identify potential disparities in CIE rates among different racial groups. The results of this study are critical for enhancing the equity of data-driven decision-making in healthcare and for addressing the racial disparities observed in COVID-19 outcomes.
BackgroundProstate cancer is the second leading cause of death among American men. If detected and treated at an early stage, prostate cancer is often curable. However, an advanced stage such as metastatic castration-resistant prostate cancer (mCRPC) has a high risk of mortality. Multiple treatment options exist, the most common included docetaxel, abiraterone, and enzalutamide. Docetaxel is a cytotoxic chemotherapy, whereas abiraterone and enzalutamide are androgen receptor pathway inhibitors (ARPI). ARPIs are preferred over docetaxel due to lower toxicity. No study has used machine learning with patients’ demographics, test results, and comorbidities to identify heterogeneous treatment rules that might improve the survival duration of patients with mCRPC. ObjectiveThis study aimed to measure patient-level heterogeneity in the association of medication prescribed with overall survival duration (in the form of follow-up days) and arrive at a set of medication prescription rules using patient demographics, test results, and comorbidities. MethodsWe excluded patients with mCRPC who were on docetaxel, cabaxitaxel, mitoxantrone, and sipuleucel-T either before or after the prescription of an ARPI. We included only the African American and white populations. In total, 2886 identified veterans treated for mCRPC who were prescribed either abiraterone or enzalutamide as the first line of treatment from 2014 to 2017, with follow-up until 2020, were analyzed. We used causal survival forests for analysis. The unit level of analysis was the patient. The primary outcome of this study was follow-up days indicating survival duration while on the first-line medication. After estimating the treatment effect, a prescription policy tree was constructed. ResultsFor 2886 veterans, enzalutamide is associated with an average of 59.94 (95% CI 35.60-84.28) more days of survival than abiraterone. The increase in overall survival duration for the 2 drugs varied across patient demographics, test results, and comorbidities. Two data-driven subgroups of patients were identified by ranking them on their augmented inverse-propensity weighted (AIPW) scores. The average AIPW scores for the 2 subgroups were 19.36 (95% CI –16.93 to 55.65) and 100.68 (95% CI 62.46-138.89). Based on visualization and t test, the AIPW score for low and high subgroups was significant (P=.003), thereby supporting heterogeneity. The analysis resulted in a set of prescription rules for the 2 ARPIs based on a few covariates available to the physicians at the time of prescription. ConclusionsThis study of 2886 veterans showed evidence of heterogeneity and that survival days may be improved for certain patients with mCRPC based on the medication prescribed. Findings suggest that prescription rules based on the patient characteristics, laboratory test results, and comorbidities available to the physician at the time of prescription could improve survival by providing personalized treatment decisions.
This study investigates potential selection bias in outcome prediction within the National COVID Cohort Collaborative (N3C) resulting from arbitrarily made decisions. In the processing of health data, decisions regarding cohort criteria and variable selection are often arbitrarily made, potentially introducing selection bias. This work explores if such decisions affect results of data analysis and potential conclusions of research studies. An experiment is conducted in which four arbitrary decisions are made. Results demonstrate significant differences in the obtained datasets and indicate a high potential for bias based on inclusion or exclusion decisions. The findings contribute to informed healthcare policies, better decision-making, and improved patient outcomes, emphasizing the necessity for testing assumptions and decisions in ongoing research that uses clinical data.
PURPOSE:To evaluate patient-level colorectal cancer outcomes in relation to residential income and racial segregation and composition of the neighborhood surrounding the diagnosing hospitals, and characterize presence of cancer-relevant diagnosis and treatment modalities that might contribute to these associations.METHODS:We utilized Georgia state cancer registry data (2010-2015), matching diagnosis information to hospital technology provided by the American Hospital Association and spatial information to the US Census. We modeled time-to-treatment and survival time, using Cox proportional hazards models, stratified by segregation. Segregation was examined as residential economic and racial evenness (Atkinson index) and isolation (isolation index) and mean income at the Census tract level. To assess possible contributing factors, analysis of hospital diagnosis and treatment technologies in relation to segregation was conducted.RESULTS:Average income of the Census tract and racial residential segregation of the diagnosing hospital's neighborhood was generally unassociated with time-to-treatment or survival time. Higher income evenness around the diagnosing hospital was associated with shorter time-to-treatment, with no association with time-to-death. Higher income isolation for the diagnosing hospital, conversely, was associated with longer times to treatment, but also longer survival times. Hospitals in regions with higher level of residential income segregation were less likely to have a particular diagnosing or treatment technologies, such as virtual colonoscopy and chemotherapy.CONCLUSION:Hospital resources may be a function of their immediate economic environment, and this may have influence on cancer outcomes. Future work should evaluate patient outcomes in light of technologies or therapies utilized within particular economic environments.
Abstract · Background: The COVID-19 pandemic has significantly stressed the healthcare system since January 2020. There are questions whether there were racial disparities in the use of resources and procedures during this period and if so, did the disparities change over the pandemic. We focus on invasive ventilation (Mechanical Ventilation/MV and Extracorporeal Membrane Oxygenation/ECMO) and racial identity of the patient for the pre-Delta and Delta timeframes. · Methods: We used data available from the National COVID Cohort Collaborative (N3C) of COVID positive patients across the US. Cox regression models were used to estimate time to MV and ECMO as the dependent variables and race, age, gender, Comorbidity index as covariates. · Results: We did not find systematic patterns of racial disparity in time to MV. Asian and Hispanic patients, but not Black patients, received MV in a delayed manner compared to White patients in the pre-Delta period. These differences were not evident in the Delta period. · Conclusions: The results show a temporal change from the pre-Delta and Delta timeframes for the time to invasive ventilation implying that any observed racial disparities improved over time. We did not find statistically significant differences in the time to ECMO across the races or over timeframes.
The growth of mobile devices coupled with advances in mobile technologies has resulted in the development and widespread use of a variety of mobile applications (apps). Mobile apps have been developed for social networking, banking, receiving daily news, maintaining fitness, and job-related tasks. The security of apps is an important concern. However, in some cases, app developers may be less interested in investing in the security of apps, if users are unwilling to pay for the added security. In this paper, we empirically examine whether consumers are less willing to pay for security features than for usability features. In addition, we examine whether a third-party certification of security features makes customers more willing to pay for security. Furthermore, we investigate the impact of risk perceptions on the willingness to pay for security. To explore these issues, we conducted a scenario-based experiment on mobile app users. Results from our analyses show that consumers are indeed less likely to pay for security features than usability features. However, the likelihood of paying for security features can be significantly increased by third-party certification of the features. Based on our analysis, we offer insights to producers of mobile apps to monetize the enhanced security features of their apps.
Prior research on IS mandates (MdIS) has mainly focused on the organization as a monolith in responding to mandates. However, companies often compete across multiple business lines and prior research has not fully examined how companies derive value from MdIS when competing across different business segments where they face different levels of customer bargaining power. Evidence of deriving value is discernible in endogenous price and demand changes as the company adheres to the mandate. Under the Medical Loss Ratio clause of the Patient Privacy and Affordable Care Act of 2010, health insurance companies were mandated to invest in information systems and associated quality improvement processes that improve the health of insurance buyers. Health insurance companies in the US operate as one or more of the three business lines serving the individual, small group/employer with less 500 employees, and the large group/employer with more than 500 employees in each state in the US. The standard and alternate translog profit functions for the financial data of US health insurance companies from 2011-2017 shed light on the impact of MdIS on price and demand decisions. The results show that the demand and prices charged across the individual and small group business lines increased with MdIS and did not change for large group business line, providing evidence of differential implementation to derive value from customers rather than passively implementing MdIS as the cost of doing business.
To evaluate time-to-treatment and survival time in colorectal cancer (CRC) patients who presumptively were not diagnosed in a hospital. Colorectal tumor-level data from Georgia Cancer Registry (GCR) was merged with American Hospital Association data for 2010–2015 using hospital identification number. Patients with tumors lacking a diagnosis hospital in the GCR were classified as presumptive non-hospital diagnosis (PNHD). Cox proportional hazard models were used to model PNHD and time-to-treatment and time-to-death following cancer diagnosis, stratified by race and controlling for personal and tumor characteristics. PNHD (n = 6,885, 29.6%) was associated with a lower likelihood of treatment at a given point in time (i.e., longer time-to-treatment), but did not differ for Black (HR = 0.77, 95% CI: 0.73, 0.82) and White (HR = 0.73, 95% CI: 0.71, 0.76) patients. Time-to-death was longer (i.e., better survival) with PNHD, which also did not differ for Black (HR = 0.70, 95% CI: 0.64, 0.76) and White (HR = 0.71, 95% CI: 0.67, 0.75) patients. These results were not explained by confounding factors or differences in tumor stage at diagnosis. These observations warrant further research to understand whether there are potentially modifiable factors associated with the diagnosing location that can be used to benefit patient treatment trajectory and survival.
Two aspects of decision-making on information security spending, executives' varying preferences for how proposals should be presented and the framing of the proposals, are developed. The proposed model of executives' commitment to information security is an interaction model (in addition to the cost of a security solution, and the risk and the potential loss of a security threat) consisting of the interaction between an executive's preferred subordinate influence approach (PSIA), rational or inspirational, and the framing, positive or negative, of a security proposal. The interaction of these two constructs affects the executive's commitment to an information security proposal. The model is tested using a scenario-based experiment that elicited responses from business executives across 100+ organizations. Results show that the interaction of the negative framing of a proposal and the inspirational PSIA of an executive affects his or her commitment to information security. Further, negative framing of a proposal and the cost of the security solution interact to decrease the executive's commitment to information security. This study underscores that prescriptions for business executives from normative models in information security spending must be complemented with appropriately framed messages to account for the differences in executives' PSIA (rational and inspirational) and cognitive biases.
Electronic Health Record (EHR) has the potential to transform the work required to create and deliver healthcare services. This has triggered large-scale adoption across hospitals. However, whether all hospitals obtain a similar effect from their EHR implementations remains an important question because there are significant differences differences in characteristics of hospitals adopting these systems. To examine differences in effects across hospitals, we examine whether the impacts of EHR applications are contingent on work domains, by assessing performance effects across hospitals with varying administrative scale and clinical complexity. Because EHRs constitute a suite of applications with different functionalities, examining the effects of different sets of applications is challenging due to nonlinear interdependencies between applications. Therefore, we use an archetype approach, identifying synergistic EHR (EHRSYN) archetype as an ideal portfolio—a conceptual anchor for a hospital’s EHR portfolio. We test contingencies by combining this technology archetype with the work domains—administrative scale and clinical complexity. We test our hypotheses using empirical data from 137 hospitals in California, hypothesizing the differences in effects of EHRs on financial, operational, and clinical performances across hospitals with different administrative scales (size) and clinical complexities (case-mix) of work. While hospitals gain the most by implementing a portfolio close to the EHRSYN archetype, our nuanced models reveal that the benefits of such portfolios increase for large hospitals and are greater for hospitals treating less complex cases. These findings underscore how variations in applications used and work domains demarcate boundaries related to the performance effects of EHRs. We present a detailed discussion of the theoretical contributions and practical implications of our findings.
Objective: To evaluate distribution of hospital-level cancer diagnosis and treatment technologies along dimensions of racial residential segregation. Study design: Cross-sectional analysis of residential segregation and availability of technologies associated with cancer diagnosis and treatment. Methods: American Hospital Association data were merged with American Community Survey data, and hospital was the unit of analysis. Isolation index and Atkinson's index were calculated for racial residential segregation for the census tract in which the hospital is located based on the composite census block groups. Logistic regression was used to model presence of cancer technologies as a function of percent below poverty (scaled 1-10), number of neighboring hospitals, and rural status. Results: Segregation measured by isolation index was associated with the availability of some technologies, independent of percentage below 125% poverty line, number of neighboring hospitals, and rural status. Diagnostic cancer technologies, such as CT scan (odds ratio [OR] = 0.928, 95% confidence interval [CI]: 0.894, 0.964), ultrasound (OR = 0.961, 95% CI: 0.927, 0.997), mammography (OR = 0.943, 95% CI: 0.914, 0.974), optical colonoscopy (OR = 0.932, 95% CI: 0.904, 0.961), and full-field digital mammography (OR = 0.948, 95% CI: 0.920, 0.977) and therapeutic cancer technology such as chemo therapy (OR = 0.963, 95% CI: 0.934, 0.992) appear to be less available in neighborhoods with higher isolation index. However, when segregation is measured by Atkinson's index, CT scan (OR = 1.064, 95% CI: 1.010, 1.121), ultrasound (OR = 1.087, 95% CI: 1.035,1.141), mammography (OR = 1.094, 95% CI: 1.049,1.141), and optical colonoscopy (OR = 1.053, 95% CI: 1.012, 1.095) are more available in neighborhoods with higher Atkinson's index. Conclusion: These results suggest that cancer diagnostic capabilities in segregated areas are in the pathway between residential segregation and cancer treatment process, and future studies should evaluate individual-level associations. (C) 2020 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved.
Background: There is a need to understand structural issues, such as differential access to or utilization of technologies or capabilities, to better understand racial disparities in cancer outcomes. The objective of this work was to evaluate time-to-treatment and survival in relation to cancer-related diagnostic and treatment technologies of the diagnosing hospital. Methods: Colorectal tumor-level data from George Cancer Registry was merged with hospital-level cancer technology data from the American Hospital Association (2010-2015). Cox proportional hazard models were used to model time-to-treatment and time-to-death following cancer diagnosis with cancer-related technologies for colon and rectosigmoid/rectal tumors, stratified by race and controlling for personal and tumor characteristics. Results: Black individuals experienced lower likelihood of treatment (HR: 0.92, 95 % CI: 0.89, 0.96) for colon tumors, but not significantly different survival (HR: 1.03, 95 % CI: 0.98, 1.09). Larger capacity or size indicators (total surgical operations, emergency room visits, and licensed beds) were associated with higher likelihood of treatment in whites, but not blacks. Higher counts of treatment related technologies were associated with better survival in whites (HR = 0.92, 95 % CI: 0.85, 0.99), but not blacks (HR: 1.07, 95 % CI: 0.95, 1.19). Virtual colonoscopy emerged as a technology related to survival favorably in whites (HR: 0.84, 95 % CI: 0.77, 0.92) and blacks (HR: 0.89, 95 % CI: 0.79, 1.00). Overall results were similar for rectosigmoid/rectal tumors as observed for colon tumors. Conclusion: The role of cancer-related technologies presence or utilization for colorectal cancer outcomes and potential racial disparities warrants further research.
The growth of mobile devices coupled with the advances in mobile technologies has resulted in the development and widespread use of a variety of mobile applications (apps). Mobile apps have been developed for social networking, banking, receiving daily news, maintaining fitness, and for job-related tasks. The security of the apps is an important concern. However, in some cases, the app developers may be less interested to invest in the security of the apps, if users are unwilling to pay for the added security. In this paper, we empirically examine whether consumers are less willing to pay for security features than for usability features. In addition, we examine whether a third-party certification of the security features makes customers more willing to pay for security. Furthermore, we investigate the impact of risk perceptions on the willingness of paying for security. To explore these issues, we conduct a scenario-based experiment of mobile app users. Results from our analyses show that, consumers are indeed less likely to pay for security features than usability features. However, the likelihood of paying for security can be significantly increased by third-party certification of the features. Based on our analysis, we offer insights to producers of mobile apps to monetize the enhanced security features of their apps.
Rajiv Kohli合作论文数Operations & Information Technology;MIS1