This Viewpoint discusses approaches to improving the measurement of patient experience and use of this information in quality improvement efforts.

BACKGROUND AND OBJECTIVES:Patient experience narratives (narratives) are an increasingly important element of both measurement approaches and improvement efforts in healthcare. Prior studies show that narratives are considered by both clinicians and staff to be an appealing, meaningful, and credible form of evidence on performance. They also suggest that making concrete use of narratives within organizational settings to improve care can be complex and challenging. Our qualitative study was designed to explore how middle managers working in a health system's outpatient clinics value and use written narratives in their day-to-day work. METHODS:We conducted qualitative interviews with 20 middle managers working in 8 outpatient clinics. Interviews were fully transcribed, loaded into MAX-QDA software, and coded using thematic analysis techniques. Code reports were extracted and reanalyzed for subthemes related to the objectives of this paper. RESULTS:Middle managers across sites described valuing narratives as a tool to: enable better patient experience assessment by augmenting data from patient experience scores; deepen understanding of and relationships with patients; provide insight about operational issues; identify areas for needed improvement and potential solutions; and facilitate strategic work. They reported using narratives for a range of activities related to their roles as supervisors, such as focusing attention on positive practices and needed improvements, promoting deeper group learning, motivating change, reinforcing sense of purpose for staff, recognizing staff strengths and training needs, and inspiring transformational thinking. Finally, interviewees reported numerous specific quality improvement projects (both short- and longer-term) that were informed by narratives-for example, by identifying an issue to be addressed or by suggesting a workable solution. Together, these interviews suggest a collective "narrative about narratives" woven by these organizational actors-a story which illustrates how narratives are highly relevant for how middle managers derive meaning from their work, put organizational values such as responsive service provision into practice, and enact their roles as supervisors. CONCLUSIONS:Our results add to the nascent literature a detailed description of how narratives can be used both as a tool for middle managers in their leadership and supervisory roles, and as a blueprint for improvement work within outpatient settings. They also illuminate why patient experience scores may improve when narrative data are collected and used. Finally, our results suggest that for middle managers, perhaps "nothing is more powerful than words" because narratives function as both an insight provider and a compelling tool that adds direction and meaning to workplace endeavors.

Policy PointsPatients' creative ideas may inform learning and innovation that improve patient-centered care.Routinely collected patient experience surveys provide an opportunity to invite patients to share their creative ideas for improvement. We develop and assess a methodological strategy that validates question wording designed to elicit creative ideas from patients.Health care organizations should consider how to report and use these data in health care delivery and quality improvement, and policymakers should consider promoting the use of narrative feedback to better understand and respond to patients' experiences.ContextLearning health systems (LHSs) have been promoted for a decade to achieve high-quality, patient-centered health care. Innovation driven by knowledge generated through day-to-day health care delivery, including patient insights, is critical to LHSs. However, the pace of translating patient insights into innovation is slow and effectiveness inadequate. This study aims to evaluate a method for systematically eliciting patients' creative ideas, examine the value of such ideas as a source of insight, and examine patients' creative ideas regarding how their experiences could be improved within the context of their own health systems.MethodsThe first stage of the study developed a survey and tested strategies for elicitation of patients' creative ideas with 600 patients from New York State. The second stage deployed the survey with the most generative open-ended question sequence within a health care system and involved analysis of 1,892 patients' responses, including 2,948 creative ideas.FindingsActionable, creative feedback was fostered by incorporating a request for transformative feedback into a sequence of narrative elicitation questions. Patients generate more actionable and creative ideas when explicitly invited to share such ideas, especially patients with negative health care experiences, those from minority racial/ethnic backgrounds, and those with chronic illness. The most frequently elicited creative ideas focused on solving challenges, proposing interventions, amplifying exceptional practices, and conveying hopes for the future.ConclusionsA valid and reliable method for eliciting creative ideas from patients can be deployed as part of routine patient experience surveys that include closed-ended survey items and open-ended narrative items in which patients share their experiences in their own words. The elicited creative ideas are promising for patient engagement and innovation efforts. This study highlights the benefits of engaging patients for quality improvement, offers a rigorously tested method for cultivating innovation using patient-generated knowledge, and outlines how creative ideas can enable organizational learning and innovation.

OBJECTIVE:To assess whether an online interactive report designed to facilitate interpretation of patients' narrative feedback produces change in ambulatory staff learning, behavior at the individual staff and practice level, and patient experience survey scores. DATA SOURCES AND SETTING:We studied 22 ambulatory practice sites within an academic medical center using three primary data sources: 333 staff surveys; 20 in-depth interviews with practice leaders and staff; and 9551 modified CG-CAHPS patient experience surveys augmented by open-ended narrative elicitation questions. STUDY DESIGN:We conducted a cluster quasi-experimental study, comparing 12 intervention and 10 control sites. At control sites, narratives were delivered free-form to site administrators via email; at intervention sites, narratives were delivered online with interactive tools for interpretation, accompanied by user training. We assessed control-versus-intervention site differences in learning, behavior, and patient experience scores. DATA COLLECTION:Staff surveys and interviews were completed at intervention and control sites, 9 months after intervention launch. Patient surveys were collected beginning 4 months pre-launch through 9 months post-launch. We used control-versus-intervention and difference-in-difference analyses for survey data and thematic analysis for interview data. PRINCIPAL FINDINGS:Interviews suggested that the interface facilitated narrative interpretation and use for improvement. Staff survey analyses indicated enhanced learning from narratives at intervention sites (29% over control sites' mean of 3.19 out of 5 across eight domains, p < 0.001) and greater behavior change at staff and practice levels (31% and 21% over control sites' means of 3.35 and 3.39, p < 0.001, respectively). Patient experience scores for interactions with office staff and wait time information increased significantly at intervention sites, compared to control sites (3.7% and 8.2%, respectively); however, provider listening scores declined 3.3%. CONCLUSIONS:Patient narratives presented through structured feedback reporting methods can catalyze positive changes in staff learning, promote behavior change, and increase patient experience scores in domains of non-clinical interaction.

BACKGROUND:Enthusiasm has grown about using patients' narratives-stories about care experiences in patients' own words-to advance organizations' learning about the care that they deliver and how to improve it, but studies confirming association have not been published. PURPOSE:We assessed whether primary care clinics that frequently share patients' narratives with their staff have higher patient experience survey scores. APPROACH:We conducted a 1-year study of 5,545 adult patients and 276 staff affiliated with nine clinics in one health system. We used multilevel models to analyze survey data from patients about their experiences and from staff about exposure to useful narratives. We examined staff confidence in own knowledge as a moderator because confidence can influence use of new information sources. RESULTS:Frequency of sharing useful narratives with staff was associated with patient experience scores for all measures, conditional on staff confidence in own knowledge ( p < .01). For operational measures (e.g., care coordination), increased sharing correlated with subsequently higher performance for more confident staff and lower performance or no difference for less confident staff, depending on measure. For relational measures (e.g., patient-provider communication), increased sharing correlated with higher scores for less confident staff and lower scores for more confident staff. CONCLUSION:Sharing narratives with staff frequently is associated with better patient experience survey scores, conditional on confidence in knowledge. PRACTICE IMPLICATIONS:Frequently sharing useful patient narratives should be encouraged as an organizational improvement strategy. However, organizations need to address how narrative feedback interacts with their staff's confidence to realize higher experience scores across domains.

Achieving patient-centered care requires reimagining aspects of care delivery and invites innovation possibilities. Eliciting creative (novel and useful) ideas from patients may enrich innovation with distinct information on experience, emotions, and needs. This research examines the potential of systematically elicited creative ideas to catalyze innovation in ambulatory care. Drawing on literature on creativity and user-based innovation, we developed and tested a survey approach to elicit patients' creative ideas for health care improvement. First, ideas extracted from 600 narratives from a representative sample were categorized thematically and analyzed for impact on closed-ended patient experience scores. Next, 1,053 narratives were collected from patients in one ambulatory care system, and we applied an innovation-focused coding schema. Patients' creative ideas addressed care experiences broadly and were significantly associated with closed-ended patient experience scores, which suggests perceived improvement gaps and idea sharing are correlated. Ideas provided information on solutions, problems, exemplars, and aspirational change, with high levels of specificity. Reframing patient participation at the beginning of the innovation process situates patients in a rare and powerful position of informing agenda setting. This approach expands our view on engagement by reimagining patients as a unique source of expertise to achieve patient-centered care.

ABSTRACT Background The Agency for Healthcare Research and Quality held a research meeting on using Consumer Assessment of Healthcare Providers and Systems (CAHPS®) data for quality improvement (QI) and evaluating such efforts. Topics Covered. Meeting addressed: 1)What has been learned about organizational factors/environment needed to improve patient experience? 2)How have organizations used data to improve patient experience? 3)What can evaluations using CAHPS data teach us about implementing successful programs to improve patient experience? Key Themes Providers and stakeholders need to be engaged early and often, standardize QI processes, complement CAHPS data with other data, and compile dashboards of CAHPS scores to identify and track improvement. Rigorous study designs are valuable, but much can be learned and accomplished through practical organization-level studies.

Given both the value of patient narratives for improving the quality of health care and the challenges of analyzing the data they contain, the authors explore the feasibility of using natural language processing to extract actionable information from these narratives.

Patient narratives have emerged as promising vehicles for making health care more responsive by helping clinicians to better understand their patients' expectations, perceptions, or concerns and encouraging consumers to engage with information about quality. A growing number of websites incorporate patients' comments. But existing comments have fragmentary content, fail to represent less vocal patients, and can be manipulated to "manage" providers' reputations. In this article, we offer the first empirical test of the proposition that patient narratives can be elicited rigorously and reliably using a five-question protocol that can be incorporated into large-scale patient experience surveys. We tested whether elicited narratives about outpatient care are complete (report all facets of patient experience), balanced (convey an accurate mix of positive and negative events), meaningful (have a coherent storyline), and representative (draw fulsome narratives from all relevant subsets of patients). The tested protocol is strong on balance and representativeness, more mixed on completeness and meaningfulness.

For two decades, various initiatives have encouraged Americans to consider quality when choosing clinicians, both to enhance informed choice and to reduce disparities in access to high-quality providers. The literature portrays these efforts as largely ineffective. But this depiction overlooks two factors: the dramatic expansion since 2010 in the availability of patients' narratives about care and the growth of information seeking among consumers. Using surveys fielded in 2010, 2014, and 2015, we assessed the impact of these changes on consumers' awareness of quality information and sociodemographic differences. Public exposure to any quality information doubled between 2010 and 2015, while exposure to patient narratives and experience surveys tripled. Reflecting a greater propensity to seek quality metrics, minority consumers remained better informed than whites over time, albeit with differences across subgroups in the types of information encountered. An education-related gradient in quality awareness also emerged over the past decade. Public policy should respond to emerging trends in information exposure, establish standards for rigorous elicitation of narratives, and assist consumers' learning from a combination of narratives and quantified metrics on clinician quality.

We conducted a simulated clinician-choice experiment, comparing choices and decision-making processes of participants (N = 688) randomized among four experimental arms: a conventional website reporting only quantitative performance information, a website reporting both qualitative (patient comments) and quantitative information, the second website augmented by a decision aid (labeling of patient comments), and the decision-aided website further augmented by the presence of a trained navigator. Introducing patient comments enhanced engagement with the quality information but led to a decline in decision quality, particularly the consistency of choices with consumers' stated preferences. Labeling comments helped erase the decline in decision quality, although the highest percentage of preference-congruent choices was seen in the navigator arm. Engagement with the quality information and satisfaction with choices available were likewise highest in the navigator arm. Findings held for high- and low-skilled decision makers. Thus, navigator assistance may be a promising strategy for equitably promoting higher quality choices in information-rich contexts.

Policy Points Narratives about patients' experiences with outpatient care are essential for quality improvement because they convey ample actionable information that both elaborates on existing domains within patient experience surveys and describes multiple additional domains that are important to patients. The content of narrative feedback from patients can potentially be translated to improved quality in multiple ways: clinicians can learn from their own patients, groups of clinicians can learn from the experience of their peers' patients, and health system administrators can identify and respond to patterns in patients' accounts that reflect systemic challenges to quality. Consistent investment by payers and providers is required to ensure that patient narratives are rigorously collected, analyzed fully, and effectively used for quality improvement.CONTEXTFor the past 25 years, health care providers and health system administrators have sought to improve care by surveying patients about their experiences. More recently, policymakers have acted to promote this learning by deploying financial incentives tied to survey scores. This article explores the potential of systematically elicited narratives about experiences with outpatient care to enrich quality improvement.METHODSNarratives were collected from 348 patients recruited from a nationally representative Internet panel. Drawing from the literature on health services innovation, we developed a two-part coding schema that categorized narrative content in terms of (a) the aspects of care being described, and (b) the actionability of this information for clinicians, quality improvement staff, and health system administrators. Narratives were coded using this schema, with high levels of reliability among the coders.FINDINGSThe scope of outpatient narratives divides evenly among aspects of care currently measured by patient experience surveys (35% of content), aspects related to measured domains but not captured by existing survey questions (31%), and aspects of care that are omitted from surveys entirely (34%). Overall, the narrative data focused heavily on relational aspects of care (43%), elaborating on this aspect of experience well beyond what is captured with communication-related questions on existing surveys. Three-quarters of elicited narratives had some actionable content, and almost a third contained three or more separate actionable elements.CONCLUSIONSIn a health policy environment that incentivizes attention to patient experience, rigorously elicited narratives hold substantial promise for improving quality in general and patients' experiences with care in particular. They do so in two ways: by making concrete what went wrong or right in domains covered by existing surveys, and by expanding our view of what aspects of care matter to patients as articulated in their own words and thus how care can be made more patient-centered. Most narratives convey experiences that are potentially actionable by those committed to improving health care quality in outpatient settings.

BackgroundPublic reports on healthcare quality typically include complex data. To lower the cognitive burden of interpreting these data, some report designers create summary, or roll-up, measures combining multiple indicators of quality into one score. Little is known about how the availability of roll-ups affects clinician choice.ObjectiveTo determine how presenting quality scores at different levels of aggregation affects patients' clinician choices.DesignWe conducted a simulated clinician-choice experiment, randomizing participants to three versions of a public reporting website and comparing their clinician choices. One version aggregated all clinician-level quality measures into roll-ups, the second provided disaggregated (drill-down) scores only, and the third offered both roll-ups and drill-downs.ParticipantsFive hundred fifty panelists drawn from a probability-based Internet panel.Main MeasuresWe assessed the amount of effort participants exerted by tracking the length of time spent on the website and the number of concrete actions taken on the website (e.g., clicking items). We evaluated decision quality by measuring whether participants selected a clinician who performed more poorly than others and incongruence between participants' stated preferences for dimensions of quality and their chosen clinician's performance on those dimensions.Key ResultsParticipants seeing drill-downs alone (mean=14.9) or with roll-ups (mean=19.2) took more actions than those who saw roll-ups alone (mean=10.5) (ps<0.05). However, participants seeing only drill-downs made poorer choices than those who saw roll-ups alone or with drill-downs. More participants seeing drill-downs chose a clinician who was outperformed (36.3% versus 23.4% [roll-up] and 25.6% [drill-down + roll-up], ps<0.05) and made choices incongruent with stated preferences (51.2% versus 45.6% [roll-up] and 47.5% [drill-down + roll-up], ps<0.05). The distinction between roll-up and drill-down was somewhat stronger for sicker participants.ConclusionsOur results suggest that roll-ups in healthcare quality reports, alone or as a complement to drill-downs, can help patients make better decisions for themselves.

Letters Health AffairsVol. 38, No. 6: Community Care For High-Need Patients Physician Information: The Authors ReplyMark Schlesinger, Lise Rybowski, and Dale Shaller AffiliationsYale School of Public Health New Haven, ConnecticutSeveryn Group Ashburn, VirginiaShaller Consulting Group Stillwater, MinnesotaPUBLISHED:June 2019Free Accesshttps://doi.org/10.1377/hlthaff.2019.00496AboutSectionsView PDFPermissions ShareShare onFacebookTwitterLinked InRedditEmail ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsDownload Exhibits TOPICSQuality of careEthnic disparitiesMedicareWe thank Asghar Shah and coauthors for suggesting some caution in interpreting the findings from our surveys of consumer exposure to physician quality information (Mar 2019). We agree that despite growing consumer engagement with comments and quality metrics, such information remains unevenly distributed across communities, often exacerbating other barriers to care.However, we do not believe that this uneven exposure detracts from the significance of our finding that nonwhite population groups were more likely to report exposure to comparative information on physician quality. First, while quantitative physician quality scores may not be available everywhere, qualitative information in the form of patients’ comments about health care providers is widely available and easily accessible online. Second, heightened exposure was evident for every minority group and every form of quality information—and in most cases, the gaps in exposure grew larger over time. Third, these gaps emerged in part because minority respondents more actively sought information on physician quality, as documented in our survey data. The heightened propensity for minority Americans to seek out information was large (odds ratio: 1.38–1.66 across specific subgroups), significant, and persisted even after other sociodemographic characteristics were controlled for.The Shahs are correct that the fact that these groups were exposed to information means only that they were aware of it—not that they were able to use it. But given that our findings controlled for differences in educational attainment and health care exposure due to serious or chronic illness, we see no reason to presume that minority consumers who actively seek information on clinician quality will be any less capable than similar white consumers of interpreting that information. Of course, being better informed does not necessarily mean being fully informed, nor does it guarantee that all patients will make equally robust choices when selecting a doctor. Loading Comments... Please enable JavaScript to view the comments powered by Disqus. DetailsExhibitsReferencesRelated Article Metrics History Published online 3 June 2019 Information© 2019 Project HOPE—The People-to-People Health Foundation, Inc.PDF downloadRelated articlesPhysician Quality Information03 Jun 2019Health Affairs

Consumers choosing a health-care provider have access to diverse information including narratives by patients about their prior experiences. However, little research has examined how narratives might improve or impede the use of information about the quality of providers’ performance. This paper describes a conceptual framework for examining mechanisms by which narrative information might influence consumer judgments and decisions about providers. We conducted a conceptual review of risk communication and behavioral decision research. We synthesized the literature to form the foundation of a conceptual framework for assessing how narrative information about provider quality impacts consumer decisions about providers. We identified four key characteristics of narratives (convey emotion; explain logic; provide relational information; and capture naturalistic experience) that may address four consumer needs (avoid surprise and regret; recognize dominant options; motivate to act or not act; and make multi-attribute tradeoff decisions). We also identified three main functions of narratives (provide a simple, powerful cue; imbue quality information with meaning; and stimulate cognition and behavior) in four decision contexts (short-term treatments; external disruptions; chronic illness; problematic experiences). A rigorous research program can be derived from the conceptual framework to generate evidence-based recommendations about whether and how patient narratives might encourage: (1) more reasoned decisions; (2) consistency with a patient’s own values/preferences; and (3) engagement with provider quality information. Research results can be used then to develop robust guidance for health communicators reporting diverse and often incommensurate performance metrics.

This document provides a detailed methodological overview of the SelectMD 2.0 provider choice experiment. This study uses an experimental design to test different methods of incorporating patient comments along with Consumer Assessment of Healthcare Providers and Services (CAHPS) survey results, Healthcare Effectiveness Data and Information Set (HEDIS)-like measures of effective clinical treatments, and indicators of patient safety in web-based physician quality reports. In addition, the study allowed for exploration of the role of patient navigators in affecting provider choice. The study aims to help the Agency for Healthcare Research and Quality (AHRQ) better understand how people choose a doctor as their regular source of medical care and advice.