Importance:Prospective clinical research studies are challenged by low enrollment rates and underrepresentation of individuals from racial and ethnic minority groups. Objective:To determine the effect of electronic message content on study enrollment among racial and ethnic groups. Design, Setting, and Participants:Four sequential randomized clinical trials (RCTs) were conducted between October 30, 2023, and November 13, 2024, in which University of Pennsylvania Health System patients (aged ≥18 years) were contacted electronically via email, text message, or both. The most effective arm from each RCT served as the control in the subsequent observational study. This analysis of all 4 RCTs was performed on the intention-to-treat principle from December 2024 through September 2025. Intervention:Messages varied by method, source, framing, and incentive structure. Main Outcomes and Measures:The primary outcome was enrollment fraction (number enrolled divided by total contacted) among Black and Hispanic participants. Overall enrollment fraction was a secondary outcome. Results:Overall, 26 215 patients were contacted and 26 029 were offered enrollment in the Penn Medicine BioBank; 63.2% were Black and 7.2% were Hispanic. Their mean (SD) age was 60.5 (17.9) years, and 60.1% were female. In the first recruitment RCT (RCT 1) (n = 8038), the enrollment fraction among 4581 Black and Hispanic patients was 0% with email, 0.5% with text, and 0.2% with email plus text (P = .06 for text vs email; P = .20 for email plus text vs email). Due to technical issues, more than 80% of participants who attempted to consent in RCT 1 failed; consent attempts were higher with text (4.8%) and email plus text (4.4%) than email alone (0.6%) (P < .001 for text vs email and email plus text vs email). In RCT 2 (n = 4271), the enrollment fraction among 1902 Black and Hispanic patients was 1.0% with research team outreach and 1.8% with clinical team outreach (P = .12). In RCT 3 (n = 3217 Black and Hispanic patients), the enrollment fraction was 3.2% with a control message, 3.6% with an appeal to altruism, and 4.1% with an appeal to social proof (P > .05 for all comparisons). In RCT 4 (n = 10 689), the enrollment fraction among 8629 Black and Hispanic patients was 2.5% with no incentive, 6.8% with $25, 5.5% with $15 plus a 5% chance at $200, 5.8% with a 5% chance at $500, and 6.5% with a 1% chance at $2500 (P < .001 for each incentive vs no incentive; P > .05 for each incentive compared with another). Conclusions and Relevance:In this series of RCTs, the enrollment fraction among Black and Hispanic patients was improved with outreach by text message and with an incentive. Trial Registration:ClinicalTrials.gov Identifier: NCT05827718.

Background More than 130 studies have relied on the ICD-9/10 code V66.7/Z51.5 for “encounter for palliative care” to identify inpatient specialty palliative care (SPC) use. One validation study among adults in an urban academic center[1] and one in a pediatric population[2] have each demonstrated poor sensitivity and inaccuracy of hospital billing codes for this purpose, but the generalizability of these findings is unknown. Objective Examine the performance of the Z51.5 hospital billing code for SPC use in diverse U.S. hospitals alone and when combined with provider billing data and identify potential sources of bias. Methods We conducted a retrospective cohort study at 1 safety-net hospital in an academic medical center (2016 – 2023) and 11 community hospitals in a national health system (2016-2018). Electronic health record and administrative data were collected for acute hospital encounters among seriously ill adults. True SPC use was defined as a completed SPC consult order in the EHR, being admitted to the PCU, or presence of an SPC provider invoice. Results Among 115,081 hospital encounters of seriously ill patients, the overall sensitivity and positive predictive value (PPV) of the Z51.5 code for any SPC use were 57.2% and 73.1%, respectively. Sensitivity increased to 91.7% for PCU encounters, 79.3% among decedents, and 83.2% for those discharged to hospice, while it was only 45.7% for those with SPC consultation alone. Performance of the Z51.5 code modestly improved when hospital and provider billing data were combined (sensitivity 70.9%, PPV 76.9%). Conclusion The Z51.5 billing code fails to capture 1 in 4 instances of inpatient SPC consultation and is highly biased towards end-of-life care. Use of ICD codes risks SPC misclassification in research studies and quality improvement initiatives that can lead to erroneous interpretation of evidence with widespread impact on future research and health system decisions regarding SPC use.

Importance:Health systems increasingly screen for health-related social needs (HRSNs), which are modifiable social factors associated with health outcomes. However, screening selection biases and HRSN burdens are poorly characterized. Objective:To evaluate patient characteristics associated with HRSN screening completion, positivity, and assistance requests. Design, Setting, and Participants:This retrospective cohort study of adults with screening-eligible outpatient or inpatient encounters in a 22-state, 92-hospital system was performed from January 1, 2020, to November 30, 2024. Data were analyzed from June 7, 2025, to February 4, 2026. Exposure:Patient sociodemographic characteristics. Main Outcomes and Measures:Unadjusted standardized mean differences and adjusted logistic regression were used to identify characteristics associated with screening completion. Multivariable regression and marginal standardization were used to identify characteristics associated with positivity for any HRSN, total HRSN burden, and assistance requests. Results:Among 1 893 331 eligible adults, 696 925 (36.8%) were 65 years or older (median age, 57 [IQR, 38-71] years), 1 138 792 (60.1%) were female, 8437 (0.4%) were American Indian or Alaska Native, 60 884 (3.2%) were Asian, 257 391 (13.6%) were Black, 4193 (0.2%) were Native Hawaiian or Other Pacific Islander, and 1 561 702 (82.5%) were White; 87 641 (4.6%) were of Hispanic or Latino ethnicity, and 916 449 (48.4%) had Medicare or Medicaid coverage. A total of 1 135 136 participants (60.0%) completed screening and 334 399 (29.5%) reported at least 1 HRSN. Unadjusted differences between screened and unscreened patients were not significant (standardized mean difference, ≤0.20). In adjusted analyses, Black patients (odds ratio [OR], 0.85; 95% CI, 0.79-0.91) and Medicaid beneficiaries (OR, 0.87; 95% CI, 0.80-0.95) had lower odds of being screened, and Hispanic or Latino patients had higher odds (OR, 1.11; 95% CI, 1.01-1.23). Outpatient underscreening was attenuated in inpatient settings for Black patients and Medicaid beneficiaries. The adjusted probabilities of any positive screen were higher among Black (absolute risk difference [ARD], 12.6%; 95% CI, 9.5%-15.7%), American Indian or Alaska Native (ARD, 10.6%; 95% CI, 9.2%-11.9%), and Native Hawaiian or Other Pacific Islander (ARD, 7.2%; 95% CI, 2.1%-12.2%) patients and among Medicare (ARD, 13.1%; 95% CI, 9.7%-16.5%) and Medicaid (ARD, 18.7%; 95% CI, 16.1%-21.2%) beneficiaries compared with White and commercially insured patients. Black patients (ARD, 9.4%; 95% CI, 6.5%-12.3%) and Medicaid beneficiaries (ARD, 16.9%; 95% CI, 16.0%-17.9%) had higher adjusted probabilities of at least 3 positive domains. Among those with at least 3 positive domains, Black patients (ARD, 21.9%; 95% CI, 16.5%-27.2%), Medicare (ARD, 7.0%; 95% CI, 5.1%-8.8%), and Medicaid (ARD, 12.2%; 95% CI, 10.9%-13.6%) beneficiaries had higher assistance requests. Conclusions and Relevance:In this cohort study across inpatient and outpatient settings, HRSN screening had a substantial reach largely free of selection. Nearly 30% of screened patients demonstrated needs, with higher rates among patients who were members of racial and ethnic minority groups and publicly insured. This study provides support for inpatient HRSN screening workflows, even as policy mandates evolve, to preserve equitable reach for patient groups with disproportionately high social needs.

Background Due to the limited specialty palliative care workforce and the growing population of adults living (longer) with serious illness in the U.S.,1,2 there is an urgent need for widespread training among generalist clinicians to improve access to high-quality primary palliative care. In preparation for a pragmatic trial testing a non-inferiority hypothesis of the overall effectiveness of trained generalist vs. specialist palliative care among more than 40,000 adults with high risk of one-year mortality in 49 U.S. hospitals across 2 health systems, we undertook a feasibility study of a Center to Advance Palliative Care (CAPC) training curriculum at two of the study hospitals. Objective Determine how to optimally engage busy clinicians to achieve high training adherence of the CAPC training curriculum. Methods The CAPC curriculum included 4 CEU-eligible courses covering broad topic areas (basics of palliative care, symptom management, and communication). Eligible clinicians were invited via email to complete training within 4-6 weeks. We employed multiple strategies to promote course completion, including identifying site champions, emails and texts to remind non-responders, and announcements at departmental meetings. Results Of 158 eligible clinicians, 100% completed the CAPC curriculum at Trinity (n=56/56) and 92% at KPSC (n=94/102) over approximately three months. Clinicians’ post-course evaluations (97% response rate) demonstrated high scores for new knowledge and relevance to practice across all 4 courses: 74% reported substantial or greater topic knowledge and there was a mean 12% increase in the proportion of clinicians rating their post-course knowledge as “very substantial” compared to pre-course. Conclusion CAPC provides an expert-developed and self-directed training platform with a full slate of courses that can be selected to meet specific leadership priorities. The strategies used to promote the training curriculum in this feasibility study led to high clinician adherence and are reproducible at scale for the planned large pragmatic trial.

Importance Prospective clinical research studies are challenged by low enrollment rates and underrepresentation of individuals from racial and ethnic minority groups. Objective To determine the effect of electronic message content on study enrollment among racial and ethnic groups. Design, Setting, and Participants Four sequential randomized clinical trials (RCTs) were conducted between October 30, 2023, and November 13, 2024, in which University of Pennsylvania Health System patients (aged ≥18 years) were contacted electronically via email, text message, or both. The most effective arm from each RCT served as the control in the subsequent observational study. This analysis of all 4 RCTs was performed on the intention-to-treat principle from December 2024 through September 2025. Intervention Messages varied by method, source, framing, and incentive structure. Main Outcomes and Measures The primary outcome was enrollment fraction (number enrolled divided by total contacted) among Black and Hispanic participants. Overall enrollment fraction was a secondary outcome. Results Overall, 26 215 patients were contacted and 26 029 were offered enrollment in the Penn Medicine BioBank; 63.2% were Black and 7.2% were Hispanic. Their mean (SD) age was 60.5 (17.9) years, and 60.1% were female. In the first recruitment RCT (RCT 1) (n = 8038), the enrollment fraction among 4581 Black and Hispanic patients was 0% with email, 0.5% with text, and 0.2% with email plus text ( P = .06 for text vs email; P = .20 for email plus text vs email). Due to technical issues, more than 80% of participants who attempted to consent in RCT 1 failed; consent attempts were higher with text (4.8%) and email plus text (4.4%) than email alone (0.6%) ( P < .001 for text vs email and email plus text vs email). In RCT 2 (n = 4271), the enrollment fraction among 1902 Black and Hispanic patients was 1.0% with research team outreach and 1.8% with clinical team outreach ( P = .12). In RCT 3 (n = 3217 Black and Hispanic patients), the enrollment fraction was 3.2% with a control message, 3.6% with an appeal to altruism, and 4.1% with an appeal to social proof ( P > .05 for all comparisons). In RCT 4 (n = 10 689), the enrollment fraction among 8629 Black and Hispanic patients was 2.5% with no incentive, 6.8% with $25, 5.5% with $15 plus a 5% chance at $200, 5.8% with a 5% chance at $500, and 6.5% with a 1% chance at $2500 ( P < .001 for each incentive vs no incentive; P > .05 for each incentive compared with another). Conclusions and Relevance In this series of RCTs, the enrollment fraction among Black and Hispanic patients was improved with outreach by text message and with an incentive. Trial Registration ClinicalTrials.gov Identifier: NCT05827718

This study uses data from a randomized clinical trial to evaluate whether use of incentives is associated with decreases in enrollment gaps among subgroup populations of race and ethnicity, sex, educational level, and age.

The traditional model specification of stepped-wedge cluster-randomized trials assumes a homogeneous treatment effect across time while adjusting for fixed-time effects. However, when treatment effects vary over time, the constant effect estimator may be biased. In the general setting of stepped-wedge cluster-randomized trials with multiple interventions, we derive the expected value of the constant effect estimator when the true treatment effects depend on exposure time periods. Applying this result to concurrent and factorial stepped wedge designs, we show that the estimator represents a weighted average of exposure-time-specific treatment effects, with weights that are not necessarily uniform across exposure periods. Extensive simulation studies reveal that ignoring time heterogeneity can result in biased estimates and poor coverage of the average treatment effect. In this study, we examine two models designed to accommodate multiple interventions with time-varying treatment effects: (1) a time-varying fixed treatment effect model, which allows treatment effects to vary by exposure time but remain fixed for each time point, and (2) a random treatment effect model, where the time-varying treatment effects are modeled as random deviations from an overall mean. In the simulations considered in this study, concurrent designs generally achieve higher power than factorial designs under a time-varying fixed treatment effect model, though the differences are modest. Finally, we apply the constant effect model and both time-varying treatment effect models to data from the Prognosticating Outcomes and Nudging Decisions in the Electronic Health Record (PONDER) trial. All three models indicate a lack of treatment effect for either intervention, though they differ in the precision of their estimates, likely due to variations in modeling assumptions.

Background: The Epic End of Life Care Index (EOLCI) predicts one-year mortality and was developed to improve serious illness care. However, prior external EOLCI evaluations had limited sample sizes, populations, and equity evaluations. In preparation for a multi-system pragmatic clinical trial, we sought to evaluate the EOLCI performance and equity in the trial's two participating health systems. Objective: Evaluate EOLCI model performance overall and across key subgroups. Design/Setting/Patients: Retrospective cohort study of patients hospitalized for at least 36 hours in 2022 to 39 hospitals in the Trinity Health and Kaiser Permanente Southern California (KPSC) health systems. Measurements: We predicted one-year mortality risk stratified by health system using the EOLCI, a logistic regression model including age, sex, race/ethnicity, ethnicity, insurance, and diagnoses. We evaluated model performance using Scaled Brier Scores (SBS; range -1 to 1; composite measures of calibration and discrimination), calibration plots, and c-statistics. Results: Among 116,749 Trinity patients with 154,063 encounters, 12,054 (10.3%) patients died within one year. Among 94,489 KPSC patients with 133,043 encounters, 16,872 (17.9%) died within one year. The SBS was -0.007 at Trinity and 0.178 at KPSC. Calibration was poor for both. Trinity's discrimination was acceptable/good (c-statistic 0.76, 95% CI 0.76-0.77), and KPSC's was good/very good (c-statistic 0.81, 95% CI 0.81-0.81). Model performance across subgroups was similar to the overall cohort. Limitations: Death data were collected exclusively within Trinity and KPSC, risking outcome misclassification; several subgroup evaluations were limited by small sample sizes. Conclusions: An external evaluation of the widely available Epic EOLCI demonstrated adequate to very good discrimination, poor calibration, and equitable performance across sociodemographic characteristics and diagnoses in two of the nation's largest health systems. Primary funding source: PCORI PLACER-2022C3-30553. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This work was supported through a Patient-Centered Outcomes Research Institute (PCORI) Award (PLACER-2022C3-30553) ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: The University of Pennsylvania Institutional Review Board approved this study I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes All data produced in the present study are available upon reasonable request to the authors

In stepped wedge cluster randomized trials (SW-CRTs), the intervention is rolled out to clusters over multiple periods. A standard approach for analyzing SW-CRTs utilizes the linear mixed model, where the treatment effect is only present after the treatment adoption, under the assumption of no anticipation. This assumption, however, may not always hold in practice because stakeholders, providers, or individuals who are aware of the treatment adoption timing (especially when blinding is challenging or infeasible) can inadvertently change their behaviors in anticipation of the forthcoming intervention. We provide an analytical framework to address the anticipation effect in SW-CRTs and study its impact. We derive expectations of the estimators based on a collection of linear mixed models and demonstrate that when the anticipation effect is ignored, these estimators give biased estimates of the treatment effect. We also provide updated sample size formulas that explicitly account for anticipation effects, exposure-time heterogeneity, or both in SW-CRTs and illustrate their impact on study power. Through simulation studies and empirical analyses, we compare the treatment effect estimators with and without adjusting for anticipation, and provide some practical considerations.

Rationale:Racial disparities in outcomes among patients with acute respiratory failure are well-described, but the contributions of clinicians to these disparities have not been evaluated. Objectives:Among mechanically ventilated patients, we evaluated racial disparities in severity of illness trajectories and adapted value-added modeling to quantify nurse and physician relationships with these disparities. Methods:In a retrospective cohort of mechanically ventilated patients across five hospitals between 2018 and 2022, we used generalized estimating equations to model the change in Laboratory-based Acute Physiology Score version 2 (LAPS) from the start to end of intensive care unit admission (Δ LAPS ). Consistent with value-added modeling, we randomly allocated the cohort into development and testing partitions, and fit separate multiple linear regression models of Δ LAPS using concurrent nurse and physician assignments (determined at 4-hour intervals), patient race, and clinician-race interaction terms as fixed effects. Clinician-specific and clinician-race interaction coefficients were extracted to determine race-specific value-add for each clinician. We defined the race-contextual value-add difference ( RCVAD ) as a clinician-level measurement of the difference in that clinician's value-add between Black and White patients in their care; a positive RCVAD indicates a more favorable severity of illness trajectory for Black relative to White patients and vice versa. Measurement and Main Results:Among 6,555 distinct patients, 7,247 clinical encounters, 405 nurses, and 70 physicians, Black patients accounted for 2,926 (40%) encounters. Overall, Black patients had significantly less improvement in Δ LAPS than White patients (difference in LAPS decline = 2.26 [0.23, 4.29], p=0.029). In the development partition, median nurse RCVAD was -0.10 (interquartile range [IQR]: -1.17, 1.14) with 191 (47%) nurses having a positive RCVAD ; median physician RCVAD was - 0.18 (IQR: -1.34, 0.56) with 29 (41%) having a positive RCVAD . Conclusions:Black mechanically ventilated patients experience less improvement in severity of illness during intensive care unit admission than White patients. While the majority of physicians and nurses were associated with disparities-exacerbating illness trajectories, many other clinicians were associated with disparities-mitigating trajectories. Future work to understand practices associated with disparities-exacerbating and disparities-mitigating care profiles could inform interventions to reduce disparities overall.

“Nudges” embedded in the electronic health record (EHR) facilitate desired decisions while preserving autonomy and may provide a scalable strategy to overcome the common implementation barrier of lack of knowledge about a best practice. We sought to test whether EHR-based nudges targeting two intensive care unit (ICU) clinician groups would safely increase evidence-based use of low tidal volume ventilation. We performed a stepped-wedge, cluster randomized, hybrid type 3 effectiveness-implementation trial in 12 ICUs from February 2021 to May 2023 to test three nudges targeting clinicians responsible for order entry and respiratory therapists responsible for operationalizing orders and documentation. A default ventilation order auto-populated a low tidal volume setting; an accountable justification order required a free-text justification to order high tidal volume; and an accountable justification flowsheet required a free-text justification to document delivery of high tidal volume. ICUs were randomly assigned to launch one of the two order nudges on a pre-specified date, followed by the flowsheet nudge six months thereafter. The primary outcome was fidelity to low tidal volume ventilation, defined as percentage of time during the first 72 h of ventilation with low tidal volumes. For additional contextual inquiry, we conducted qualitative interviews with ICU clinicians regarding their perspectives on low tidal volume ventilation and study nudges. The primary analysis included 4412 patients. Unadjusted median fidelity to low tidal volume ventilation was 45.7 https://clinicaltrials.gov/study/NCT04663802

In orderto assess the relevance of trial results or the appropriate trials methods, interest holders (eg, clinicians, patients, and policy makers) need a clear description of the trial's research question. To improve clarity and consistency in clinical trials, the International Conference on Harmonisation (ICH) released the ICH E9(R1) addendum, which set out a framework for defining estimands-a precise description of the treatment effect to be estimated. While the ICH E9(R1) addendum has been widely adopted, it primarily focused on individually randomised trials. In contrast, cluster randomised trials, where groups of individuals are randomised, present additional challenges for defining estimands. Therefore, the CRT-Estimands Frameworkwas developed as a consensus based extension of the ICH E9(R1) addendum for cluster randomised trials. This framework provides a set of attributes that should be described when defining estimands in cluster randomised trials, with the objective of improving the clarity of the estimands, and consequently, the research questions, in these trials. This article presents the CRT-Estimands Framework with explanations and examples of how it can be implemented. Adopting the framework will improve the clarity of estimands in cluster randomised trials and facilitate interest holders to make informed decisions from these trials.

Rationale: Patients with sepsis and/or acute respiratory failure are at high risk for death or long hospital stays, yet limited evidence exists to guide triage to intensive care units (ICUs) or general medical wards for the majority of these patients who do not initially require life support. Objectives: To identify factors that influence how hospitals triage patients with capacity-sensitive conditions and those factors that may account for observed ICU relative to ward, or ward relative to ICU, benefits for such patients. Methods: We conducted an explanatory sequential mixed-methods study. As part of a 27-hospital, two-health system retrospective cohort study, we calculated hospital-specific measurements of ICU net benefit for patients with sepsis and/or acute respiratory failure. Hospitals among the highest ICU net benefit and lowest ICU net benefit (or highest ward net benefit) from each study health system were selected for in-depth qualitative study. At each hospital, interviews were conducted with emergency department, ward, and ICU clinicians and administrators. Interview transcripts were analyzed using flexible coding and the framework method. Results: Interviews were conducted with 118 respondents (46 physicians, 43 nurses, 5 advanced practice providers, and 24 administrators) from four hospitals. Respondents across hospitals agreed that the prediction of patient trajectory is central to triage decisions, but there was variation in opinion across work locations about optimal pretriage emergency department interventions in terms of intensity, repetition, clinical reassessment, and observation duration. The main difference observed between high and low ICU net benefit hospitals related to the way respondents working in the ICU and ward described their responses to patients who experience rapid clinical deviations from triage-expected trajectories, including sustained lack of critical care needs after admission to the ICU and acute critical care needs after admission to the ward. Hospitals with low ICU net benefit (or high ward net benefit) had particularly robust and proactive rapid response and clinical decompensation surveillance practices for ward-admitted patients. Conclusions: Particularly proactive rapid response programs that deliver on-location critical care may quantitatively increase ward net benefit by bringing ICU benefits without ICU-associated harms to ward patients who become critically ill.

To examine the association of admission NICU capacity strain with neonatal mortality and morbidity. 2008–2021 South Carolina cohort using linked vital statistics and discharge data of 22–44 weeks GA infants, born at hospitals with ≥ level 2 unit and ≥5 births <34 weeks GA/year. The exposure was deciles of admission capacity strain, defined as the sum of infants ≤44 weeks GA with a congenital anomaly plus infants <34 weeks GA. The primary outcome was a composite of mortality and term and preterm complications. We used Poisson generalized linear mixed models to examine the association of exposure with outcome adjusting for patient and hospital characteristics. We studied 64,647 infants from 30 hospitals. High capacity strain was associated with increased risk of mortality and morbidity adjusting for patient/hospital factors (for example, tenth decile aIRR 1.14, 95% CI 1.03–1.27). Capacity strain is associated with adverse NICU outcomes.

BACKGROUND Tobacco use has a disproportionate impact on older, medically underserved adults. Mobile health (mHealth) tools hold promise for increasing reach of treatment options, yet introduce new barriers to access and use. RESEARCH QUESTION How can investigators incorporate patient and community input into the design and testing of accessible, scalable, and equity-promoting mHealth tobacco treatment tools? STUDY DESIGN AND METHODS We present a model for mHealth tobacco treatment tool development using a longitudinal community-partnered design process. We iteratively developed and refined tools used in a large, pragmatic trial. First, a stakeholder advisory committee (SAC) convened with members including individual patients and representatives from patient and health equity advocacy groups, community and government public health services, clinical program leads, and health system and insurance leaders. Second, we conducted a patient needs assessment to confirm or expand on SAC recommendations using semistructured interviews among patients meeting ≥ 1 medically underserved criteria who smoked tobacco daily. Transcribed interviews were coded and analyzed for patterns of patients’ desired design elements. RESULTS The SAC recommended key strategies to promote cultural relevance of the tools, maximize engagement of participants, and prevent attrition, which were incorporated into the intervention and trial design. To further refine the approach, we completed interviews with 39 patients from November 2020 to September 2021. Many respondents used telemedicine tools with their clinicians yet were skeptical of their use for tobacco treatment due to lack of facility with mobile technologies. Patients recommended direct support options, avoidance of novel smartphone applications, and customizable features. INTERPRETATION We provide a model for patient-centered design that incorporates community engagement through longitudinal advisors and wider representation of patients. Longitudinal community engagement that incorporates broad patient perspectives facilitates effective development and deployment of mHealth tools to maximize responsiveness to patient and community needs.

Importance Interest in the use of prediction models to support referrals to palliative care is surging. Few high-performing models have been developed, implemented, and pilot-tested following responsible artificial intelligence (AI) principles and transparent reporting guidelines. Objective Collaborate with physicians and clinical informaticists to plan, develop, validate, implement, and pilot-test a high-performing, statistically fair prediction model—the Serious Illness Clinical Indicator (SICLI)—to support palliative care referrals following responsible AI principles. Design Cohort study of hospitalizations occurring from January 1st, 2022, to December 31st, 2022. Setting Kaiser Permanente Southern California Participants Patients 18+ years, not admitted to maternity or psychiatry departments and hospitalized for at least 36 hours. Exposures We augmented a commercial prediction model, the Epic End-of-Life-Care Index (EOLCI), by adding locally available predictors, including co-morbidity and laboratory acuity risk scores, nursing flow sheet and health care utilization data. Main Outcomes and Measures We predicted 12-month mortality risk using a split-sample design, least absolute shrinkage, and selection operator (LASSO) using logistic regression, and five-fold cross-validation. Performance was assessed via the Area Under the Receiver Operating Curve (AUC) and calibration plots. The final model was chosen by clinicians and informaticists for its balance of performance and parsimony. Responsible AI principles guided each development step. Results Twelve-month mortality rate among 133,043 hospitalizations of at least 36 hours was 23%. Patients who died were older (76±13 vs. 64±17 years) and had higher co-morbidity burden (Charlson, 4.49±3.14 vs. 2.06±2.44). SICLI was very well calibrated and outperformed the EOLCI (AUC of 0.87 (95% CI:0.86, 0.87) vs. 0.81, (95% CI: (0.80, 0.81)). SICLI’s high and very-high risk group (probability cut-off 0.6-<95 and >=0.95), produced positive predictive values (PPVs) of 72.0% and 94.0%, respectively. SICLI was implemented in KPSC’s Electronic Health Record. Post-implementation validation by a palliativist against the 12-month surprise question via review of 25 charts yielded a PPV of 100% and 80% for the very-high and high-risk groups. We report on model documentation, socialization, and governance. Conclusion and Relevance SICLI is an equitable, high-performing mortality prediction model that builds on and outperforms the EOLCI. It was implemented and pilot-tested in a large health care system. Question Can we build on a commercially available mortality risk score to develop, implement, and pilot-test a high-performing prediction model to support palliative care decision-making in a large integrated health care system? Findings The Serious Illness Clinical Indicator (SICLI) augments and outperforms the Epic End-of-Life Care Score (EOLCI) and was implemented and pilot-tested using a responsible AI framework. Meaning Predictive models can be valuable tools to support palliative care referrals. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement This work was supported by the Kaiser Permanente Southern California Care Improvement Research Team (PIs: Drs. Nau and Nguyen) and the Patient Centered Outcomes Research Institute (PLACER-2022C3-30553, PIs: Drs. Halpern and Courtright). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: This study was approved by the Kaiser Permanente Southern California (13886) and University of Pennsylvania (855378) Institutional Review Boards. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes The data are available for use through collaboration with the KPSC study investigators under the conditions of sufficient funding by the requestor and data use agreements between all institutions that govern data access, storage, and short- and long-term use. Qualified researchers trained in human subject confidentiality protocols interested in collaborating with the KPSC study team can contact Dr. Claudia Nau (claudia.l.nau{at}kp.org).

Persons living with dementia (PWD) and their family care partners may prioritize possible treatment outcomes differently. We sought to understand how PWD and their family care partners value hypothetical treatment outcomes relative to one other using a point allocation task. Participants distributed 100 points across six potential outcomes based on perceived relative importance. Outcomes included slowed decline in brain function, reduced agitation and combativeness, reduced family caregiver stress, prolonged independence in performing activities of daily living, increased time living at home, and increased socialization and engagement. Equality of distributions between outcome scores and between subgroups were compared using Wilcoxon signed-rank and rank-sum tests, respectively. To ensure comprehension and understanding of rationale for point distribution, we engaged participants in a “think out loud” approach as they completed the point allocation task. Qualitative data was documented in field notes and reviewed by the research team independently and during consensus meetings to identify relevant themes. 201 individuals participated ( n  = 154 [77%] care partners and n  = 47 [23%] PWD; Table 1). Across all participants, slowing declines in brain function scored highest (20 [IQR 10, 30], p  < .05 for all pair-wise comparisons). Care partners compared to patients more highly valued reduced agitation and combativeness (15.5 [IQR 10, 25] vs 10 [IQR 3, 16)], p  < .001). Patients compared to care partners more highly valued reducing family caregiver stress (16 [IQR 10, 30] vs 10 [IQR 5, 20], p  = .003). Among care partners, those whose loved one had mild dementia more highly valued slowing declines in brain function (20 [IQR 10, 37.5] vs 15 [IQR 10, 25], p  = .02). Conversely, care partners of those with moderate or severe dementia more highly valued reducing agitation and combativeness (20 [IQR 10, 25] vs 12.5 [IQR 7.5, 21] p  = 0.007). Qualitative analysis revealed key themes regarding participant rationale for prioritizing certain outcomes over others (Table 2). PWD and dementia care partners expressed differing values when considering treatment outcomes. The development of future treatments should incorporate measurement of a range of valued outcomes critical to patients across the spectrum of impairment and their families.

Persons living with dementia (PWD) and their family care partners may have different priorities when considering new dementia treatments. We sought to determine the relative importance of several different outcome domains when considering choosing a hypothetical dementia treatment. We conducted a discrete choice experiment (DCE) among PWD and family care partners administered online and facilitated by a trained research coordinator. Participants were recruited from memory centers in Pennsylvania, Wisconsin, and Michigan. In each DCE task, participants were asked to choose one of two hypothetical dementia treatments that would produce variable benefits across multiple outcome domains. The full DCE, conducted with care partners, consisted of eight forced-choice tasks assessing respondent preferences within six attributes: brain function, distressing symptoms, physical function, home time, family caregiver stress, and socialization and engagement. Participating PWD were shown an abbreviated version of the DCE consisting of six tasks with a random subset of four of the six attributes. Data were analyzed using mixed effects logistic regression models to assess how each attribute influenced likelihood of selecting a treatment option. Participant comprehension, reasoning, and reactions to the DCE were solicited throughout and documented in field notes. Field notes were subsequently analyzed using constant comparison techniques to identify relevant themes. We surveyed 201 participants (154 care partners; 47 PWD; Table 1). Across all participants, the highest priority outcomes were relief in distressing symptoms and increasing home time (OR=0.66, 95% CI=0.61-0.71 and OR=0.66, 95% CI=0.61-0.71). When comparing utilities across PWD and care partners, PWD more highly valued increasing home time, improving physical function, reducing family caregiver stress, and increasing patient socialization and engagement compared to family care partners (Figure). Qualitative analysis revealed key themes regarding participant decision making processes and attitudes towards the DCE experience (Table 2). Increasing home time is an important treatment outcome for PWD and family care partners. Prioritization of other treatment outcomes vary across these key stakeholder subgroups. Participation in a DCE assessing treatment outcomes as a communication process was valued by patients and care partners, suggesting its potential utility as a communication aid or values elicitation tool.

RATIONALE:Prospective clinical research studies are essential for determining the effectiveness and safety of drugs, medical devices, and healthcare delivery interventions. However, low enrollment, particularly among Black and Hispanic patients, challenges the generalizability of results and fairness of research. Leveraging insights from behavioral economics to modify the content of messages recruiting patients to join research studies may increase enrollment and representativeness of trial populations. PRIMARY HYPOTHESIS:Method of outreach, source of outreach, message framing, and financial incentives will have important effects on enrollment fraction of Black and Hispanic patients electronically approached for participation in a prospective clinical research study. DESIGN:ITERATE (NCT05827718) is a series of 4 randomized clinical trials (RCTs) designed to rigorously, systematically, and iteratively test the effects of different messaging strategies informed by behavioral economic theory on the enrollment of Black and Hispanic individuals into the Penn Medicine BioBank (PMBB), a prospective registry. For all 4 RCTs, we will identify patients eligible for enrollment in the PMBB (those with ≥ 1 encounter with the University of Pennsylvania Health System in the past 3 months, a phone number able to receive text messages or a valid email address on file, no history of consenting to or declining enrollment in the PMBB, and able to provide their own consent) and randomly assign them to receive different outreach messages. RCT 1 will test the method of outreach (email vs. text message vs. email + text message); RCT 2, source of outreach (research team vs. clinical team); RCT 3, message framing (appeal to altruism vs. appeal to social proof vs. control); and RCT 4, financial incentive (none vs. medium guarantee vs. small guarantee + small lottery vs. medium lottery vs. large lottery). In each RCT, at least 50% of the participants will be Black or Hispanic. The primary outcome of each RCT is enrollment fraction, defined as the number of participants who enroll in the PMBB divided by the total number of participants who received an outreach message, compared between arms among both Black and Hispanic patients. Secondary outcomes will include overall enrollment fraction and enrollment fraction among White patients. The "winning" strategies in earlier RCTs will be incorporated as the "standard of care" in the subsequent RCTs.