INTRODUCTION:Fidelity assessments can support healthcare services to deliver care consistent with best practises. However, early psychosis (EP) fidelity assessment tools typically require a volume of service data that is often unavailable to small or new programmes. In this study, we pilot a formative fidelity assessment approach to address these challenges. METHODS:A formative assessment approach to using the First Episode Psychosis Services-Fidelity Scale (FEPS-FS) was developed to enable the assessment of small and new EP programmes. Over 48 months, EPI-CAL EP learning health care network programmes completed standard FEPS-FS fidelity assessments, formative assessments for new programmes, or formative assessments for small programmes, depending upon programme eligibility. RESULTS:Of 27 remote fidelity assessments completed with EP programmes across California, nine (33.3%) had insufficient service data to complete a standard FEPS-FS assessment. Utilising the proposed formative assessment approach, one programme met the criteria for a new programme assessment, and seven for a small programme assessment. In the new programme assessment approach, 34 of 36 items could be assessed. In the small programme assessments, a median of 19 items was scored, with a mean FEPS-FS score range from 3.45 to 4.12. CONCLUSION:These findings suggest that a formative approach to fidelity assessment can generate a meaningful amount of data, capture known variability between programmes, and potentially identify areas for service improvement to enhance quality for small and new programmes. However, critical data regarding the delivery of pharmacological and psychosocial care were not captured, highlighting the limitations of the approach.

OBJECTIVE:This study aimed to develop and pilot the Clinical High Risk for Psychosis Services Fidelity Scale (CHRPS-FS). METHODS:A literature review was conducted to identify evidence-based treatments for individuals at clinical high risk for psychosis (CHRP). These findings were compared with the First-Episode Psychosis Services Fidelity Scale (FEPS-FS). Common items were retained, and others were added, modified, or deleted. Next, the Delphi process was conducted with 17 clinical and academic experts in CHRP care to determine consensus on the importance and validity of each item. Concurrently, the preliminary tool was piloted in eight coordinated specialty care (CSC) clinics serving individuals with CHRP. RESULTS:The literature review identified two components of CHRP care that were not detailed in the FEPS-FS and were added to the CHRPS-FS; furthermore, one FEPS-FS item was modified and six were removed. In the Delphi process, clinical and academic experts achieved a consensus of >80% in two rounds, with some changes in item wording and the addition of one item (stepped care approach). A CHRPS-FS assessment was successfully piloted in eight CSC clinics. The mean CHRPS-FS rating score was 3.96 (range 3.75-4.23), and the median proportion of items rated at good to high fidelity was 72% (range 66%-78%). CONCLUSIONS:The CHRP-FS is feasible to implement, has face validity based on expert consensus, can be completed in conjunction with a FEPS-FS assessment or alone, and captures variability across programs. The CHRPS-FS measures service delivery and is suitable for clinical trials, learning health care systems, and quality improvement efforts.

BackgroundThe Early Psychosis Intervention Network of California project, a learning health care network of California early psychosis intervention (EPI) programs, prioritized incorporation of community partner feedback while designing its eHealth app, Beehive. Though eHealth apps can support learning health care network data collection aims, low user acceptance or adoption can pose barriers to successful implementation. Adopting user-centered design (UCD) approaches, such as incorporation of user feedback, prototyping, iterative design, and continuous evaluation, can mitigate these potential barriers. ObjectiveWe aimed to use UCD during development of a data collection and data visualization web-based and tablet app, Beehive, to promote engagement with Beehive as part of standard EPI care across a diverse user-base. MethodsOur UCD approach included incorporation of user feedback, prototyping, iterative design, and continuous evaluation. This started with user journey mapping to create storyboards, which were then presented in UCD workshops with service users, their support persons, and EPI providers. We incorporated feedback from these workshops into the alpha version of Beehive, which was also presented in a UCD workshop. Feedback was again incorporated into the beta version of Beehive. We provided Beehive training to 4 EPI programs who then piloted Beehive’s beta version. During piloting, service users, their support persons, and EPI program providers completed Beehive surveys at enrollment and every 6 months after treatment initiation. To examine preliminary user acceptance and adoption during the piloting phase, we assessed rates of participant enrollment and survey completion, with a particular focus on completion of a prioritized survey: the Modified Colorado Symptom Index. ResultsUCD workshop feedback resulted in the creation of new workflows and interface changes in Beehive to improve the user experience. During piloting, 48 service users, 42 support persons, and 72 EPI program providers enrolled in Beehive. Data were available for 88% (n=42) of service users, including self-reported data for 79% (n=38), collateral-reported data for 42% (n=20), and clinician-entered data for 17% (n=8). The Modified Colorado Symptom Index was completed by 54% (n=26) of service users (total score: mean 24.16, SD 16.81). In addition, 35 service users had a support person who could complete the Modified Colorado Symptom Index, and 56% (n=19) of support persons completed it (mean 26.71, SD 14.43). ConclusionsImplementing UCD principles while developing the Beehive app resulted in early workflow changes and produced an app that was acceptable and feasible for collection of self-reported clinical outcomes data from service users. Additional support is needed to increase collateral-reported and clinician-entered data.

INTRODUCTION:Early identification and intervention for psychosis-spectrum experiences can improve long-term outcomes. However, treatment-seeking in the United States is often delayed, with factors like stigma, potentially heightened for individuals reporting psychotic-like experiences (PLEs). METHODS:Participants (N = 1904) self-selected into a survey of mental health care utilization after seeking out online psychosis screening through Mental Health America. Chi-square analyses compared high and low PLEs groups on MHCU attitudes and processes, including perceived need for care. RESULTS:Significantly more individuals with high PLEs (H-PLEs) endorsed a need to address their mental health than low-PLEs individuals (L-PLEs; p < .001). The H-PLEs group was more likely to report having a plan to utilize mental health care (MHC; p < .001, OR = 1.41), but reported lower confidence in their ability to seek MHC than L-PLEs (p = .003, Cohen's d = 0.14). H-PLEs respondents were more likely to report confusion about engaging in MHC and feeling too overwhelmed to take action compared to L-PLEs (p < .001, OR = 1.53; p < .001, OR = 1.76, respectively). Qualitative data (n = 145) suggested that the H-PLEs group was more likely to experience emotional barriers to MHCU than the L-PLEs group, such as feeling uncomfortable, scared, or hopeless (p < .05). CONCLUSIONS:Results suggest that individuals with H-PLEs may have different attitudes and barriers to seeking MHC than individuals with L-PLEs. Interventions to address lower confidence and difficulties navigating MHC processes, in addition to studies with more representative samples, are recommended.

OBJECTIVE:Time between the onset of psychosis and the start of treatment significantly influences outcomes. Rapid access to care is essential, yet barriers such as stigma, difficulties with navigating the mental health system, and financial constraints prolong this process. This mixed-methods study aimed to assess how these barriers affect participation in early psychosis services. METHODS:A directed content analysis of telephone log data was conducted from intake assessments at an early psychosis clinic. Stepwise logistic regression and analyses of variance were used to evaluate the impact of barriers on assessment completion and time from referral to assessment. RESULTS:Of 1,048 individuals screened for early psychosis services, 201 completed a telephone assessment. Individuals who dropped out had a higher proportion of barriers overall than did those who completed the assessment (p<0.01). Greater than 50% of interactions included at least one barrier, with logistical issues being the most common. Increased barriers were correlated with longer assessments and lower completion rates. Adults and Hispanic participants reported more barriers, compared with adolescents and non-Hispanic individuals, respectively. Significant contributors to nonengagement included unknown gender, public insurance, and various barriers. CONCLUSIONS:Identifiable barriers to intake assessment were frequently reported by clients and were associated with higher intake noncompletion and a longer assessment process. Efforts to address logistical barriers may represent an essential step in improving the linkage process and reducing the duration of untreated psychosis.

Despite the substantial capacity of qualitative and mixed methods research to advance healthcare and interventions knowledge, most large-scale health intervention trials exclusively use quantitative methods. The authors argue that qualitative research can optimize investments in these studies. As researchers within the Early Psychosis Intervention Network (EPINET), the authors highlight examples of how qualitative research has enhanced this national initiative, organizing them with a Learning Health System (LHS) framework to demonstrate the ways qualitative research can increase value at each phase of a health trial. They emphasize the critical need for integrating qualitative research from the beginning of health trials, ensuring its influence in decision-making, creating infrastructure to support it, and promoting meaningful representation within research teams. By illustrating the advantages of qualitative research in EPINET, they advocate for sustained commitment to qualitative research in health trials to maximize value in client and provider experience, cost, and population health.

Using data collected in routine care delivery to inform treatment is a key feature of a learning health system (LHS). In this study, we explored the experiences of service users and providers adopting measurement-based care (MBC) in early psychosis (EP) specialty care settings. Qualitative interviews were conducted with 32 providers and 12 service users across 18 programs in the Early Psychosis Intervention Network of California (EPI-CAL). These findings were compared with quantitative data from Beehive, EPI-CAL's data collection and review application. Regarding the clinical benefits of MBC in EP, three broad themes were identified - supporting safety monitoring and response, the assessment process, and delivery of psychotherapy. Outside of direct clinical care, Beehive was considered to support clinical supervision and external reporting, while service users reported data collection facilitated self-reflection. In the quantitative Beehive application data collected from 23 EP programs, high utilization of the safety alert system was evident (349 alerts in total, of which 338 [96.85 %] were resolved at a median of 2.03 days). However, service users' key survey data was only reviewed by assigned providers in 32.22 % (142 of 441) of cases. While providers and service users saw many benefits to Beehive, utilization was highly inconsistent outside of the alert system. Going forward, further consideration of how best to support EP providers to consistently use data in care is necessary to maximize the utility of the LHS approach and positively impact outcomes.

BACKGROUND:A longer duration of untreated psychosis (DUP) is associated with poorer treatment outcomes. Screening for psychosis spectrum disorders in the primary care setting could help support the earlier detection and treatment of individuals in need. However, the acceptability of screening for psychosis in this setting as part of routine care is currently unknown. METHODS:We conducted a qualitative interview study with providers and service users who participated in an early psychosis screening program conducted in an integrated behavioral health primary care (IBH-PC) setting. Interviews were recruited from one of eight WellSpace Federally Qualified Health Center IBH-PC clinics in the Sacramento, CA area. Transcripts of the recorded interviews were analyzed using thematic analysis. RESULTS:In total, 12 providers and eight service users participated in the interviews. Most service user and provider participants were supportive of psychosis screening in an IBH-PC setting, but not as part of the general practitioner consultation due to the brief, non-behavioral health nature of many of the appointments, and the expected low prevalence of psychosis in this population. The support of leadership, adequate training and support, staff turnover, and organizational changes were all seen to impact the successful implementation of the program. Different barriers and facilitators were considered important at each stage of the process from introducing the screening procedures to service users; to determining when, where, and how to screen; and how to effectively manage the referral and post-referral stages. CONCLUSIONS:Despite the additional challenges of screening in an IBH-PC setting relative to secondary mental health services, the process was considered acceptable and feasible to providers and service users. Services that plan to conduct psychosis screening in their clinics need to consider the challenges and their potential solutions to implementation at each stage of the screening process.

Screening for psychosis spectrum disorders in primary care could improve early identification and reduce the duration of untreated psychosis. However, the accuracy of psychosis screening in this setting is unknown. To address this, we conducted a diagnostic accuracy study of screening for psychosis spectrum disorders in eight behavioral health services integrated into primary care clinics. Patients attending an integrated behavioral health appointment at their primary care clinic completed the Prodromal Questionnaire - Brief (PQ-B) immediately prior to their intake assessment. This was compared to a diagnostic phone interview based on the Structured Interview for Psychosis Risk Syndromes (SIPS). In total, 145 participants completed all study procedures, of which 100 screened positive and 45 negative at a provisional PQ-B threshold of ≥20. The PQ-B was moderately accurate at differentiating psychosis spectrum from no psychosis spectrum disorders; a PQ-B distress score of ≥27 had a sensitivity and specificity of 71.2 % and 57.0 % respectively. In total, 66 individuals (45.5 %) met criteria for a psychosis spectrum disorder and 24 (16.7 %) were diagnosed with full psychosis, indicating a high prevalence of psychosis in the sample. Overall, screening for psychosis spectrum disorders in an IBH primary care setting identified a relatively high number of individuals and may identify people that would otherwise be missed. The PQ-B performed slightly less well than in population-based screening in community mental health settings. However, the findings suggest this may represent an effective way to streamline the pathway between specialty early psychosis programs and primary care clinics for those in need.

Objective: Learning health care networks can significantly improve the effectiveness, consistency, and cost-effectiveness of care delivery. As part of a data harmonization process, incorporation of the perspectives of community partners to maximize the relevance and utility of the data is critical. Methods: A mixed-methods focus group study was conducted with early psychosis program providers, leadership, service users, and family members to explore their priorities regarding data collection in early psychosis care. Focus group transcripts were analyzed through thematic analysis. Results: Twenty-two focus groups comprising 178 participants were conducted across 10 early psychosis programs. Participants considered functioning, quality of life, recovery, and symptoms of psychosis as key outcomes to assess, although variation by participants' roles was also evident. Participants emphasized the clinical utility of assessing a broad range of predictors of care outcomes, favored a broad conceptualization of the constructs assessed, and indicated a preference for client-reported measures. Participants also emphasized the importance of surveys adopting a recovery-oriented, strengths-based approach. Conclusions: Large-scale aggregation of health care data collected as part of routine care offers opportunities for research and may have a positive impact on care delivery and quality improvement activities. However, these benefits are contingent on the data being both relevant and accessible to those who deliver and receive such care. This study highlights an approach that may inform the development of core assessment batteries used, optimizing the utility of such data for all community partners.

Objective: Expanded funding to support care across the crisis continuum is intended to improve behavioral health outcomes. A greater understanding of how to effectively implement and integrate local crisis care systems has been identified as a research and policy priority. The aim of this study was to explore provider perceptions of the barriers and facilitators associated with implementing effective behavioral health crisis services. Methods: The authors conducted semistructured qualitative interviews with personnel from 15 behavioral health crisis care programs across California. Purposive sampling was used to ensure adequate representation of peer specialists, clinicians, mental health workers, and program leaders. Interview transcripts were analyzed via an inductive approach to thematic analysis. On the basis of patterns identified in the data, initial codes were developed, reviewed, and combined into overarching preliminary themes and subthemes. Results: Twenty-nine crisis care personnel participated. Facilitators of effective crisis care included an optimal crisis service structure, a client-centered approach, engagement with clients' support systems, and collaboration with community partners to link clients to services and enable safe delivery of crisis care. Barriers at the client, program, and system levels were identified, with solutions proposed for each. Conclusions: The participants identified features of crisis care that could improve program implementation and effectiveness or could help mitigate identified barriers. As states and local municipalities work to implement an integrated system of care across the crisis care continuum, input from frontline providers can be used to support the development of new programs, refine existing services, and inform future directions for research.

BackgroundA prolonged first episode of psychosis (FEP) without adequate treatment is a predictor of poor clinical, functional, and health outcomes and significant economic burden. Team-based "coordinated specialty care" (CSC) for early psychosis (EP) has established effectiveness in promoting clinical and functional recovery. However, California's CSC program implementation has been unsystematic and could benefit from standardizing its processes and data collection infrastructure. To address this, we established a consortium of EP clinics across the state via a Learning Health Care Network (LHCN) framework to develop the Early Psychosis Intervention Network of California (EPI-CAL). EPI-CAL's LHCN developed a core battery of evidence-based measures for service users and family members and linked them together using a unique data collection and visualization application, Beehive.Methods and objectivesEPI-CAL's LHCN collects, visualizes, and aggregates data at the individual and clinic level for EP programs across California via Beehive. Beehive was designed to: (1) collect outcomes data from service users receiving care at EP programs and their support persons, (2) provide the data to providers on a secure web-based dashboard to support measurement-based care, and (3) allow data to be used for program or research analysis. We will (1) determine the feasibility of implementing an LHCN across a diverse, decentralized network of early psychosis programs, (2) determine if the implementation of an LHCN increases the delivery of measurement-based care, and (3) determine if the implementation of measurement-based care is associated with significant improvements in key service user outcomes. EPI-CAL's network will contribute data to the Early Psychosis Intervention Network (EPINET) program.DiscussionThe current study aims to establish an LHCN of EP clinics in California that implements harmonized data collection using Beehive and assesses the feasibility of establishing such a network. Our goal is for this harmonized data collection approach to be used to inform decisions and develop learning opportunities for service users, staff, and administrators, and to improve outcomes for service users and their supporters in CSC care. Further, the data will enable programs and research teams to examine what elements of care lead to program success and improved treatment outcomes for service users.Clinical trials registrationwww.ClinicalTrials.gov, identifier NCT04007510; registered 07/05/2019.

The EPI-CAL project, a learning health care network (LHCN) of California early psychosis intervention (EPI) programs, prioritized incorporation of community partner feedback while designing its eHealth application, Beehive. Though eHealth applications can support LHCN data collection aims, low user acceptance or adoption can pose barriers to successful implementation. Adopting user-centered design approaches, such as incorporation of user feedback and continuous evaluation, can mitigate these potential barriers. We employed user-centered design during development of a data-collection and data-visualization eHealth application, Beehive, to promote engagement with Beehive as part of standard EPI care across a diverse user-base. We conducted user-centered design workshops with service users, their support persons, and EPI providers during Beehive storyboarding and alpha testing. We incorporated feedback from these workshopsinto the beta version of the application. Then, after receiving training, four EPI programs piloted Beehive’s beta version. During piloting, service users and their primary support persons (PSPs) completed Beehive surveys at enrollment and every 6 months after treatment initiation. To examine preliminary user acceptance and adoption during the piloting phase, we assessed rates of participant enrollment and survey completion, with a particular focus on the Modified Colorado Symptom Index (MCSI). User-centered design workshop feedback resulted in the creation of new workflows and interface changes in Beehive to improve the user experience. During piloting, 48 service users, 42 PSPs, and 72 EPI program providers enrolled in Beehive. Data is available for 88% (n=42) of service users, including self-reported data for 79% (n=38), collateral-reported data for 42% (n=20), and clinician-entered data for 17% (n=8). The MCSI was completed by 54% (n=26) of service users (total score: M=24.16, SD=16.81 and 56% (n=19) of support persons (M=26.71, SD=14.43). Implementing user-centered designed while developing the Beehive application resulted in early development workflow changes and produced an application that was acceptable and feasible for collection of self-reported clinical outcomes data from service users. Additional support is needed to increase collateral-reported and clinician-entered data. NCT04007510

Background Increased use of eHealth technology and user data to drive early identification and intervention algorithms in early psychosis (EP) necessitates the implementation of ethical data use practices to increase user acceptability and trust. Objective First, the study explored EP community partner perspectives on data sharing best practices, including beliefs, attitudes, and preferences for ethical data sharing and how best to present end-user license agreements (EULAs). Second, we present a test case of adopting a user-centered design approach to develop a EULA protocol consistent with community partner perspectives and priorities. Methods We conducted an exploratory, qualitative, and focus group–based study exploring mental health data sharing and privacy preferences among individuals involved in delivering or receiving EP care within the California Early Psychosis Intervention Network. Key themes were identified through a content analysis of focus group transcripts. Additionally, we conducted workshops using a user-centered design approach to develop a EULA that addresses participant priorities. Results In total, 24 participants took part in the study (14 EP providers, 6 clients, and 4 family members). Participants reported being receptive to data sharing despite being acutely aware of widespread third-party sharing across digital domains, the risk of breaches, and motives hidden in the legal language of EULAs. Consequently, they reported feeling a loss of control and a lack of protection over their data. Participants indicated these concerns could be mitigated through user-level control for data sharing with third parties and an understandable, transparent EULA, including multiple presentation modalities, text at no more than an eighth-grade reading level, and a clear definition of key terms. These findings were successfully integrated into the development of a EULA and data opt-in process that resulted in 88.1% (421/478) of clients who reviewed the video agreeing to share data. Conclusions Many of the factors considered pertinent to informing data sharing practices in a mental health setting are consistent among clients, family members, and providers delivering or receiving EP care. These community partners’ priorities can be successfully incorporated into developing EULA practices that can lead to high voluntary data sharing rates.

Importance:Reducing the duration of untreated psychosis (DUP) is essential to improving outcomes for people with first-episode psychosis (FEP). Current US approaches are insufficient to reduce DUP to international standards of less than 90 days.Objective:To determine whether population-based electronic screening in addition to standard targeted clinician education increases early detection of psychosis and decreases DUP, compared with clinician education alone.Design, Setting, and Participants:This cluster randomized clinical trial included individuals aged 12 to 30 years presenting for services between March 2015 and September 2017 at participating sites that included community mental health clinics and school support and special education services. Eligible participants were referred to the Early Diagnosis and Preventative Treatment (EDAPT) Clinic. Data analyses were performed in September and October 2019 for the primary and secondary analyses, with the exploratory subgroup analyses completed in May 2021.Interventions:All sites in both groups received targeted education about early psychosis for health care professionals. In the active screening group, clients also completed the Prodromal Questionnaire-Brief using tablets at intake; referrals were based on those scores and clinical judgment. In the group receiving treatment as usual (TAU), referrals were based on clinical judgment alone.Main Outcomes and Measures:Primary outcomes included DUP, defined as the period from full psychosis onset to the date of the EDAPT diagnostic telephone interview, and the number of individuals identified with FEP or a psychosis spectrum disorder. Exploratory analyses examined differences by site type, completion rates between conditions, and days from service entry to telephone interview.Results:Twenty-four sites agreed to participate, and 12 sites were randomized to either the active screening or TAU group. However, only 10 community clinics and 4 school sites were able to fully implement population screening and were included in the final analysis. The total potentially eligible population size within each study group was similar, with 2432 individuals entering at active screening group sites and 2455 at TAU group sites. A total of 303 diagnostic telephone interviews were completed (178 [58.7%] female individuals; mean [SD] age, 17.09 years [4.57]). Active screening sites reported a significantly higher detection rate of psychosis spectrum disorders (136 cases [5.6%], relative to 65 [2.6%]; P < .001) and referred a higher proportion of individuals with FEP and DUP less than 90 days (13 cases, relative to 4; odds ratio, 0.30; 95% CI, 0.10-0.93; P = .03). There was no difference in mean (SD) DUP between groups (active screening group, 239.0 days [207.4]; TAU group 262.3 days [170.2]).Conclusions and Relevance:In this cluster trial, population-based technology-enhanced screening across community settings detected more than twice as many individuals with psychosis spectrum disorders compared with clinical judgment alone but did not reduce DUP. Screening could identify people undetected in US mental health services. Significant DUP reduction may require interventions to reduce time to the first mental health contact.Trial Registration:ClinicalTrials.gov Identifier: NCT02841956.