Objective: Counselor assessment of suicide risk is one key component of crisis counseling, and standards require risk assessment in every crisis counseling conversation. Efforts to increase risk assessment frequency are limited by quality improvement tools that rely on human evaluation of con- versations, which is labor intensive, slow, and impossible to scale. Advances in machine learning (ML) have made pos- sible the development of tools that can automatically and immediately detect the presence of risk assessment in crisis counseling conversations. Methods: To train models, a coding team labeled every statement in 476 crisis counseling calls (193,257 statements) for a core element of risk assessment. The authors then fine-tuned a transformer-based ML model with the labeled data, utilizing separate training, validation, and test data sets. Results: Generally, the evaluated ML model was highly consistent with human raters. For detecting any risk as- sessment, ML model agreement with human ratings was 98% of human interrater agreement. Across specific labels, average F1 (the harmonic mean of precision and recall) was 0.86 at the call level and 0.66 at the statement level and often varied as a result of a low base rate for some risk labels. Conclusions: ML models can reliably detect the presence of suicide risk assessment in crisis counseling conversations, presenting an opportunity to scale quality improvement efforts.

Importance Use of asynchronous text-based counseling is rapidly growing as an easy-to-access approach to behavioral health care. Similar to in-person treatment, it is challenging to reliably assess as measures of process and content do not scale. Objective To use machine learning to evaluate clinical content and client-reported outcomes in a large sample of text-based counseling episodes of care. Design, Setting, and Participants In this quality improvement study, participants received text-based counseling between 2014 and 2019; data analysis was conducted from September 22, 2022, to November 28, 2023. The deidentified content of messages was retained as a part of ongoing quality assurance. Treatment was asynchronous text-based counseling via an online and mobile therapy app (Talkspace). Therapists were licensed to provide mental health treatment and were either independent contractors or employees of the product company. Participants were self-referred via online sign-up and received services via their insurance or self-pay and were assigned a diagnosis from their health care professional. Exposure All clients received counseling services from a licensed mental health clinician. Main Outcomes and Measures The primary outcomes were client engagement in counseling (number of weeks), treatment satisfaction, and changes in client symptoms, measured via the 8-item version of Patient Health Questionnaire (PHQ-8). A previously trained, transformer-based, deep learning model automatically categorized messages into types of therapist interventions and summaries of clinical content. Results The total sample included 166 644 clients treated by 4973 therapists (20 600 274 messages). Participating clients were predominantly female (75.23%), aged 26 to 35 years (55.4%), single (37.88%), earned a bachelor’s degree (59.13%), and were White (61.8%). There was substantial variability in intervention use and treatment content across therapists. A series of mixed-effects regressions indicated that collectively, interventions and clinical content were associated with key outcomes: engagement (multiple R = 0.43), satisfaction (multiple R = 0.46), and change in PHQ-8 score (multiple R = 0.13). Conclusions and Relevance This quality improvement study found associations between therapist interventions, clinical content, and client-reported outcomes. Consistent with traditional forms of counseling, higher amounts of supportive counseling were associated with improved outcomes. These findings suggest that machine learning–based evaluations of content may increase the scale and specificity of psychotherapy research.

Background The opioid epidemic has resulted in expanded substance use treatment services and strained the clinical workforce serving people with opioid use disorder. Focusing on evidence-based counseling practices like motivational interviewing may be of interest to counselors and their supervisors, but time-intensive adherence tasks like recording and feedback are aspirational in busy community-based opioid treatment programs. The need to improve and systematize clinical training and supervision might be addressed by the growing field of machine learning and natural language-based technology, which can promote counseling skill via self- and supervisor-monitoring of counseling session recordings. Methods Counselors in an opioid treatment program were provided with an opportunity to use an artificial intelligence based, HIPAA compliant recording and supervision platform (Lyssn.io) to record counseling sessions. We then conducted four focus groups—two with counselors and two with supervisors—to understand the integration of technology with practice and supervision. Questions centered on the acceptability of the clinical supervision software and its potential in an OTP setting; we conducted a thematic coding of the responses. Results The clinical supervision software was experienced by counselors and clinical supervisors as beneficial to counselor training, professional development, and clinical supervision. Focus group participants reported that the clinical supervision software could help counselors learn and improve motivational interviewing skills. Counselors said that using the technology highlights the value of counseling encounters (versus paperwork). Clinical supervisors noted that the clinical supervision software could help meet national clinical supervision guidelines and local requirements. Counselors and clinical supervisors alike talked about some of the potential challenges of requiring session recording. Conclusions Implementing evidence-based counseling practices can help the population served in OTPs; another benefit of focusing on clinical skills is to emphasize and hold up counselors’ roles as worthy. Machine learning technology can have a positive impact on clinical practices among counselors and clinical supervisors in opioid treatment programs, settings whose clinical workforce continues to be challenged by the opioid epidemic. Using technology to focus on clinical skill building may enhance counselors’ and clinical supervisors’ overall experiences in their places of work.

Implementation of behavior therapy for ADHD faces challenges in community settings. We describe development of a community-based implementation model for adolescent ADHD behavior therapy (Supporting Teens' Autonomy Daily; STAND) blended with Motivational Interviewing (MI). A stakeholder-engaged development approach is used based on the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Resulting model features include: (a) task-shifting supervision from experts to agency supervisors, (b) holding bi-weekly technical assistance drop-ins to provide training and implementation supports, (c) MI integrity monitoring and feedback by artificial intelligence (AI), (d) AI-generated metrics for STAND content fidelity, (e) digitizing resources (manual, worksheets, tips, videos) on a clinician dashboard, (f) creating visual displays of feedback using badges and graphs, and (g) adding a rapport-building session prior to manualized content. We conducted stakeholder focus groups (N = 32) and two pilot studies to evaluate the new STAND AI measurement tool and revised service-delivery model (N = 6 therapists, 7 youth and parents, 3 agency supervisors). Results revealed advantages and disadvantages of the model, supported the promise of a STAND AI fidelity measurement tool, and indicated initial feasibility, acceptability, and agency engagement in STAND's community-based implementation model. We discuss future directions for continued iterative development and testing. Video examples are included as supplementary material.

BACKGROUNDSubstance use disorders (SUDs) are highly prevalent and associated with a substantial public health burden. Although evidence-based interventions exist for treating SUDs, many individuals remain symptomatic despite treatment, and relapse is common.Mindfulness-based interventions (MBIs) have been examined for the treatment of SUDs, but available evidence is mixed.OBJECTIVESTo determine the effects of MBIs for SUDs in terms of substance use outcomes, craving and adverse events compared to standard care, further psychotherapeutic, psychosocial or pharmacological interventions, or instructions, waiting list and no treatment.SEARCH METHODSWe searched the following databases up to April 2021: Cochrane Drugs and Alcohol Specialised Register, CENTRAL, PubMed, Embase, Web of Science, CINAHL and PsycINFO. We searched two trial registries and checked the reference lists of included studies for relevant randomized controlled trials (RCTs).SELECTION CRITERIARCTs testing a MBI versus no treatment or another treatment in individuals with SUDs. SUDs included alcohol and/or drug use disorders but excluded tobacco use disorders. MBIs were defined as interventions including training in mindfulness meditation with repeated meditation practice. Studies in which SUDs were formally diagnosed as well as those merely demonstrating elevated SUD risk were eligible.DATA COLLECTION AND ANALYSISWe used standard methodological procedures expected by Cochrane.MAIN RESULTSForty RCTs met our inclusion criteria, with 35 RCTs involving 2825 participants eligible for meta-analysis. All studies were at high risk of performance bias and most were at high risk of detection bias. Mindfulness-based interventions (MBIs) versus no treatment Twenty-four RCTs included a comparison between MBI and no treatment. The evidence was uncertain about the effects of MBIs relative to no treatment on all primary outcomes: continuous abstinence rate (post: risk ratio (RR) = 0.96, 95% CI 0.44 to 2.14, 1 RCT, 112 participants; follow-up: RR = 1.04, 95% CI 0.54 to 2.01, 1 RCT, 112 participants); percentage of days with substance use (post-treatment: standardized mean difference (SMD) = 0.05, 95% CI -0.37 to 0.47, 4 RCTs, 248 participants; follow-up: SMD = 0.21, 95% CI -0.12 to 0.54, 3 RCTs, 167 participants); and consumed amount (post-treatment: SMD = 0.10, 95% CI -0.31 to 0.52, 3 RCTs, 221 participants; follow-up: SMD = 0.33, 95% CI 0.00 to 0.66, 2 RCTs, 142 participants). Evidence was uncertain for craving intensity and serious adverse events. Analysis of treatment acceptability indicated MBIs result in little to no increase in study attrition relative to no treatment (RR = 1.04, 95% CI 0.77 to 1.40, 21 RCTs, 1087 participants). Certainty of evidence for all other outcomes was very low due to imprecision, risk of bias, and/or inconsistency. Data were unavailable to evaluate adverse events. Mindfulness-based interventions (MBIs) versus other treatments (standard of care, cognitive behavioral therapy, psychoeducation, support group, physical exercise, medication) Nineteen RCTs included a comparison between MBI and another treatment. The evidence was very uncertain about the effects of MBIs relative to other treatments on continuous abstinence rate at post-treatment (RR = 0.80, 95% CI 0.45 to 1.44, 1 RCT, 286 participants) and follow-up (RR = 0.57, 95% CI 0.28 to 1.16, 1 RCT, 286 participants), and on consumed amount at post-treatment (SMD = -0.42, 95% CI -1.23 to 0.39, 1 RCT, 25 participants) due to imprecision and risk of bias. The evidence suggests that MBIs reduce percentage of days with substance use slightly relative to other treatments at post-treatment (SMD = -0.21, 95% CI -0.45 to 0.03, 5 RCTs, 523 participants) and follow-up (SMD = -0.39, 95% CI -0.96 to 0.17, 3 RCTs, 409 participants). The evidence was very uncertain about the effects of MBIs relative to other treatments on craving intensity due to imprecision and inconsistency. Analysis of treatment acceptability indicated MBIs result in little to no increase in attrition relative to other treatments (RR = 1.06, 95% CI 0.89 to 1.26, 14 RCTs, 1531 participants). Data were unavailable to evaluate adverse events.AUTHORS' CONCLUSIONSIn comparison with no treatment, the evidence is uncertain regarding the impact of MBIs on SUD-related outcomes. MBIs result in little to no higher attrition than no treatment. In comparison with other treatments, MBIs may slightly reduce days with substance use at post-treatment and follow-up (4 to 10 months). The evidence is uncertain regarding the impact of MBIs relative to other treatments on abstinence, consumed substance amount, or craving. MBIs result in little to no higher attrition than other treatments. Few studies reported adverse events.

Emotional distress is a common reason for seeking psychotherapy, and sharing emotional material is central to the process of psychotherapy. However, systematic research examining patterns of emotional exchange that occur during psychotherapy sessions is often limited in scale. Traditional methods for identifying emotion in psychotherapy rely on labor-intensive observer ratings, client or therapist ratings obtained before or after sessions, or involve manually extracting ratings of emotion from session transcripts using dictionaries of positive and negative words that do not take the context of a sentence into account. However, recent advances in technology in the area of machine learning algorithms, in particular natural language processing, have made it possible for mental health researchers to identify sentiment, or emotion, in therapist–client interactions on a large scale that would be unattainable with more traditional methods. As an attempt to extend prior findings from Tanana et al. ( 2016 ), we compared their previous sentiment model with a common dictionary-based psychotherapy model, LIWC, and a new NLP model, BERT. We used the human ratings from a database of 97,497 utterances from psychotherapy to train the BERT model. Our findings revealed that the unigram sentiment model (kappa = 0.31) outperformed LIWC (kappa = 0.25), and ultimately BERT outperformed both models (kappa = 0.48).

Efforts to help therapists improve their multicultural competence (MCC) rely on measures that can distinguish between different levels of competence. MCC is often assessed by asking clients to rate their experiences with their therapists. However, differences in client ratings of therapist MCC do not necessarily provide information about the relative performance of therapists and can be influenced by other factors including the client's own characteristics. In this study, we used a repeated measures design of 8,497 observations from 1,458 clients across 35 therapists to clarify the proportion of variability in MCC ratings attributed to the therapist versus the client and better understand the extent that an MCC measure detects therapist differences. Overall, we found that a small amount of variability in MCC ratings was attributed to the therapist (2%) and substantial amount attributed to the client (70%). These findings suggest that our measure of MCC primarily detected differences at the client level versus therapist level, indicating that therapist MCC scores were largely dependent on the client. Clinical implications and recommendations for future MCC research and measurement are discussed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Efforts to improve the implementation of evidence-based treatments (EBT) have recently made important strides. One such example is understanding the vital role that weekly consultation plays as therapists learn to deliver an EBT. Because mechanism-based research can further support EBT implementation, the present study sought to examine the potential relationship between therapist self-efficacy in relation to treatment fidelity and outcomes. We examined therapist self-efficacy ratings from 80 therapists working with 188 patients. These data were collected as part of a randomized controlled implementation trial testing cognitive processing therapy (CPT). Across post-workshop training conditions, we ran multilevel models to assess (1) changes in therapist self-efficacy, (2) therapist self-efficacy in relation to treatment fidelity, and (3) therapist-self-efficacy in relation to patient PTSD symptom outcomes. We found that therapist self-efficacy significantly improved over the course of 6 months of CPT training. Baseline therapist self-efficacy was differentially associated with client outcomes based on post-workshop training condition. Specifically, therapists with low self-efficacy that did not receive post-workshop consultation tended to have poorer outcomes than therapists with low self-efficacy that received consultation. In the present sample, therapist self-efficacy was not related to treatment fidelity. As this was the first study to examine therapist self-efficacy in the implementation of an evidence-based treatment, our findings suggest that self-efficacy may be an important implementation factor in treatment outcomes and worthy of ongoing research.

The current meta-analysis examined the effects of psilocybin in combination with behavioral interventions on anxiety and depression in samples with elevated symptoms. Across four studies (one uncontrolled; three randomized, placebo-controlled; N = 117), within-group pre-post and pre-follow-up effects on anxiety and depression were large (Hedges' gs=1.16 to 1.47) and statistically significant. Across three placebo-controlled studies, pre-post placebo-controlled effects were also large (gs = 0.82 to 0.83) and statistically significant. No serious adverse events were reported. Limitations include the small number of studies and risk for bias within studies. Results tentatively support future research on psilocybin for the treatment of anxiety and depression.

Artificial intelligence generally and machine learning specifically have become deeply woven into the lives and technologies of modern life. Machine learning is dramatically changing scientific research and industry and may also hold promise for addressing limitations encountered in mental health care and psychotherapy. The current paper introduces machine learning and natural language processing as related methodologies that may prove valuable for automating the assessment of meaningful aspects of treatment. Prediction of therapeutic alliance from session recordings is used as a case in point. Recordings from 1,235 sessions of 386 clients seen by 40 therapists at a university counseling center were processed using automatic speech recognition software. Machine learning algorithms learned associations between client ratings of therapeutic alliance exclusively from session linguistic content. Using a portion of the data to train the model, machine learning algorithms modestly predicted alliance ratings from session content in an independent test set (Spearman's ρ = .15, p < .001). These results highlight the potential to harness natural language processing and machine learning to predict a key psychotherapy process variable that is relatively distal from linguistic content. Six practical suggestions for conducting psychotherapy research using machine learning are presented along with several directions for future research. Questions of dissemination and implementation may be particularly important to explore as machine learning improves in its ability to automate assessment of psychotherapy process and outcome. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Direct observation of psychotherapy and providing performance-based feedback is the gold-standard approach for training psychotherapists. At present, this requires experts and training human coding teams, which is slow, expensive, and labor intensive. Machine learning and speech signal processing technologies provide a way to scale up feedback in psychotherapy. We evaluated an initial proof of concept automated feedback system that generates motivational interviewing quality metrics and provides easy access to other session data (e.g., transcripts). The system automatically provides a report of session-level metrics (e.g., therapist empathy) and therapist behavior codes at the talk-turn level (e.g., reflections). We assessed usability, therapist satisfaction, perceived accuracy, and intentions to adopt. A sample of 21 novice (n = 10) or experienced (n = 11) therapists each completed a 10-min session with a standardized patient. The system received the audio from the session as input and then automatically generated feedback that therapists accessed via a web portal. All participants found the system easy to use and were satisfied with their feedback, 83% found the feedback consistent with their own perceptions of their clinical performance, and 90% reported they were likely to use the feedback in their practice. We discuss the implications of applying new technologies to evaluation of psychotherapy. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Motivational interviewing (MI) theory proposes a process whereby a set of therapist behaviors has direct effects on client outcomes and indirect effects through in-session processes (e.g., client change talk). Despite clear empirical support for the efficacy of MI across settings, the results of studies evaluating proposed links between MI process and outcome have been less clear. In the present study, we used a series of multivariate meta-analyses to test whether there are differential relationships between specific MI-consistent and MI-inconsistent therapist behaviors, MI therapist global ratings, client change language, and clinical outcomes. Based on 19 primary studies (N = 2,614), we found a significant relationship between MI-consistent therapist behaviors and greater client change talk, as well as greater client sustain talk. Higher therapist global ratings (empathy and MI spirit) were significantly related to increased MI-consistent behaviors, decreased MI-inconsistent behaviors, increased client change talk, yet also increased client sustain talk. Therapist global ratings were not significantly related to clinical outcomes. Client sustain talk was a significant predictor of worse clinical outcomes, while client change talk was unrelated to outcome. Variability within the correlations indicated that MI-consistent and MI-inconsistent therapist behaviors were differentially related to therapist global ratings of empathy and MI spirit. Similar to past research, present findings provide equivocal support for hypothesized MI process outcome relationships. Clinical implications and future areas of MI mechanism research are discussed. (PsycINFO Database Record

Providers' adherence in the delivery of behavioral interventions for substance use disorders is not fixed, but instead can vary across sessions, providers, and intervention sites. This variability can substantially impact the quality of intervention that clients receive. However, there has been limited work to systematically evaluate the extent to which substance use intervention adherence varies from session-to-session, provider-to-provider, and site-to-site. The present study quantifies the extent to which adherence to Motivational Interviewing (MI) for alcohol and drug use varies across sessions, providers, and intervention sites and compares the extent of this variability across three common MI research contexts that evaluate MI efficacy, MI effectiveness, and MI training. Independent raters coded intervention adherence to MI from 1275 sessions delivered by 216 providers at 15 intervention sites. Multilevel models indicated that 57%–94% of the variance in MI adherence was attributable to variability between sessions (i.e., within providers), while smaller proportions of variance were attributable to variability between providers (3%–26%) and between intervention sites (0.1%–28%). MI adherence was typically lowest and most variable within contexts evaluating MI training (i.e., where MI was not protocol-guided and delivered by community treatment providers) and, conversely, adherence was typically highest and least variable in contexts evaluating MI efficacy and effectiveness (i.e., where MI was highly protocolized and delivered by trained therapists). These results suggest that MI adherence in efficacy and effectiveness trials may be substantially different from that obtained in community treatment settings, where adherence is likely to be far more heterogeneous.

OBJECTIVE:Three recent meta-analyses have made the claim, albeit with some caveats, that cognitive-behavioral treatments (CBT) are superior to other psychotherapies, in general or for specific disorders (e.g., social phobia).METHOD:The purpose of the present article was to examine four issues in meta-analysis that mitigate claims of CBT superiority: (a) effect size, power, and statistical significance, (b) focusing on disorder-specific symptom measures and ignoring other important indicators of psychological functioning, (c) problems inherent in classifying treatments provided in primary studies into classes of treatments, and (d) the inclusion of problematic trials, which biases the results, and the exclusion of trials that fail to find differences among treatments.RESULTS:When these issues are examined, the effects demonstrating the superiority of CBT are small, nonsignificant for the most part, limited to targeted symptoms, or are due to flawed primary studies.CONCLUSION:Meta-analytic evidence for the superiority of CBT in the three meta-analysis are nonexistent or weak.

For decades, psychologists have emphasized the provision of multiculturally competent psychotherapy to reduce racial and ethnic disparities in mental health treatment. However, the relationship between multicultural competencies (MC) and other measures of clinical process and treatment outcome has shown heterogeneity in effect sizes. This meta-analysis tested the association of client ratings of therapist MC with measures of therapeutic processes and outcome, including: (a) working alliance, (b) client satisfaction, (c) general counseling competence, (d) session impact, and (e) symptom improvement. Among 18 studies (20 independent samples) included in the analysis, the correlation between therapist MC and outcome (r = .29) was much smaller than the association with process measures (r = .75), but there were no significant differences in correlations across different types of MC or clinical process measures. Providing some evidence of publication bias, effect sizes from published studies (r = .67) were larger than those from unpublished dissertations (r = .28). Moderator analyses indicated that client age, gender, the representation of racial-ethnic minority (R-EM) clients, and clinical setting were not associated with effect size variability. Based on these findings, we discuss implications and recommendations for future research that might lead to a better understanding of the effects of therapist MC on treatment process and outcome. Primary needs in future research include the development and evaluation of observer ratings of therapist MC and the implementation of longitudinal research designs.

Although evidence suggests that the benefits of psychodynamic treatments are sustained over time, presently it is unclear whether these sustained benefits are superior to non-psychodynamic treatments. Additionally, the extant literature comparing the sustained benefits of psychodynamic treatments compared to alternative treatments is limited with methodological shortcomings. The purpose of the current study was to conduct a rigorous test of the growth of the benefits of psychodynamic treatments relative to alternative treatments across distinct domains of change (i.e., all outcome measures, targeted outcome measures, non-targeted outcome measures, and personality outcome measures). To do so, the study employed strict inclusion criteria to identify randomized clinical trials that directly compared at least one bona fide psychodynamic treatment and one bona fide non-psychodynamic treatment. Hierarchical linear modeling (Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2011) was used to longitudinally model the impact of psychodynamic treatments compared to non-psychodynamic treatments at post-treatment and to compare the growth (i.e., slope) of effects beyond treatment completion. Findings from the present meta-analysis indicated that psychodynamic treatments and non-psychodynamic treatments were equally efficacious at post-treatment and at follow-up for combined outcomes (k=20), targeted outcomes (k=19), non-targeted outcomes (k=17), and personality outcomes (k=6). Clinical implications, directions for future research, and limitations are discussed.

Due to seemingly mixed empirical results, questions persist about the possible role of deployments and combat exposure. We conducted a narrative review and meta-analysis of 22 published studies to integrate findings regarding the relationship of deployment-related predictors (i.e., deployment, deployment to a combat zone, combat experience, and exposure to specific combat events) with suicide-related outcomes (i.e., suicide ideation, attempt, and death). Across all predictors and outcomes, the combined effect was small and positive, r = .08 [0.04, 0.13], and marked by significant heterogeneity, I2  = 99.9%, Q(21)=4880.16, p < .0001, corresponding to a 25% increased risk for suicide-related outcomes among those who have deployed. Studies examining the relationship between exposure to killing and atrocities (k = 5) showed the largest combined effect, r = .12 [0.08, 0.17], and less heterogeneity, I2  = 84.4%, Q(4)=34.96, p < .0001, corresponding to a 43% increased risk for suicide-related outcomes among those exposed to killing or atrocity. Implications for theory, research, and clinical practice are discussed.