Opportunities and challenges to using a cross-sectional, patient-reported experience measure of shared decision making to evaluate the impact of an intervention centered on serious illness conversations in oncology.

317 Background: Patient-reported experience measures (PREMs) are an important quality metric for healthcare delivery, even more so for the assessment of the quality of serious illness conversations (SIC) in oncology. We deployed a validated, 3-item PREM for shared decision-making (SDM), CollaboRATE, as part of the assessment for an intervention aimed at increasing the conduct of routine SIC in the outpatient oncology clinic. The use of CollaboRATE sought to identify patient-reported differences in SDM potentially attributable to this intervention. Methods: During the 12-month project period, CollobaRATE was included in outpatient oncology experience surveys (PressGaney) sent to patients who attended visits at the main campus. Surveyed patients were selected at random, mailed the survey, and were not resurveyed on the same clinician for a minimum of 6 months. We used a top score method for our primary outcome (e.g., selecting the highest mark on all three CollaboRATE items). Incomplete survey items were scored below the top score. We analyzed survey results for all patients and those identified as SIC-eligible by a clinician (i.e., radio button function within the electronic health record (EHR)). We also stratified results by SIC intervention versus non-SIC intervention clinicians. We used a chi-square test of independence with a statistical significance threshold of p≤0.05 to assess any association. Results: From 8/1/2021 to 7/31/2022, we received 1309 CollaboRATE surveys representing an overall response rate of 25%. During this period, SIC intervention clinicians represented 19% (8/43) of the oncology clinicians (e.g. MD/DO, APRN, PA) practicing at this site, and compromised 19% (247/1309) of outpatient surveys collected. Among all surveyed cancer patients, the proportion selecting the top score for all items was not significantly different at 73% (181/247) for SIC intervention clinicians versus 68% (720/1062) for non-SIC intervention clinicians (p=0.09). Among patients identified as SIC-eligible in the EHR, the proportion selecting top-box scores for all items was statistically significant at 75% (97/130) for SIC intervention clinicians versus 59% (31/53) for non-SIC intervention clinicians (p=0.031). Conclusions: CollaboRATE has been used in primary care to benchmark clinician and clinic performance in SDM, however, its use as an assessment strategy in serious illness communication in oncology is unique. Caution should be made when interpreting the SIC-eligible patient analysis, as the control group is small (i.e. likely undercounted) given the reliance on clinicians to identify patients as SIC-eligible within the EHR to be included. The collection and analysis of PREMs for SDM were feasible in an outpatient oncology setting.