Point and level (P & L) systems are commonly used for behavior management and modification in youth residential treatment centers. In 2019, the Association of Children's Residential Centers (ACRC) released a position paper urging youth residential programs to abolish prescriptive, universally applied point and level systems because they are inconsistent with the principles of trauma-sensitive care and can be counterproductive to treatment. Despite this, few residential agencies have executed this change, possibly due to the complexities of dismantling long-held practices, concerns about youth safety, and lack of knowledge about other trauma-sensitive approaches. In this case study, we describe how one agency de-implemented its universal P & L system across three residential campuses, replacing it with the Neurosequential Model of Therapeutics (NMT) and Collaborative Problem Solving (CPS). An analysis of administrative data before and after the removal of P & L suggests no evidence of increased safety risk for youth or staff. We discuss the agency's strategies for de-implementation and the determinants that helped and hindered the process and provide recommendations for other residential agencies seeking to make this change.

Background:Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings.Methods:Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers' self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers' accuracy as a function of their global fidelity.Results & Conclusions:The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.

Collaborative Problem Solving (CPS) is an intervention for reducing children’s challenging behaviors. The aim of the present study was to evaluate the effects of family therapy using CPS in an outpatient clinic that specializes in treating children with challenging behaviors. One hundred and twenty families presented for treatment. Diagnoses at intake were varied, and 100 children (83%) had symptoms that were in the clinical range at intake. Parents reported significant change in their understanding of challenging behavior and prediction of children’s behavioral symptoms 3 months into treatment. Furthermore, children’s improvement was predicted by their parents’ increased understanding that cognitive skill deficits are responsible for challenging behavior. These results suggest that using CPS in community-based, outpatient family treatment is effective for helping children who exhibit a range of clinical symptoms. Results provide insights for clinical practice and research on CPS.

Collaborative Problem Solving (CPS) is a widely disseminated, neurobiologically based, trauma-sensitive treatment for children's challenging behavior. Measuring treatment integrity is critical to support implementation and continued research on the approach. This article presents the development and psychometric evaluation of an observational system, the CPS Manualized Expert-Rated Integrity Coding System (CPS-MEtRICS), for measuring CPS treatment integrity. Audio recordings of in-home treatment sessions (159 CPS and 82 treatment-as-usual) were independently rated by pairs of seven trained coders for integrity and the affective aspect of client-provider relationship. Results showed satisfactory interrater reliability (intraclass correlations [ICCs] 0.64-0.86). Additionally, the integrity scores on the CPS-MEtRICS discriminated between providers delivering CPS and treatment-as-usual, while these two groups did not differ in general client-provider affective bond, supporting the construct and discriminant validity of ratings using the system. This study provided evidence for the reliability of the CPS-MEtRICS and the validity of its integrity ratings, which can be used for research on CPS and which will provide the foundation for other CPS integrity measurement tools useful in practice settings. This article also provides a model that can be used when developing integrity measurement systems for other therapeutic approaches. Public Significance Statement This study provides psychometric support for a system that can be used to measure whether practitioners are using the Collaborative Problem Solving (CPS) treatment approach with integrity, or as intended by the developers of the approach. Researchers and practitioners can use this system to advance their skill and understanding of CPS, and its development provides a model to support the creation of integrity measurement systems for other therapeutic approaches.

Twenty-one parents participated in a Collaborative Problem Solving (CPS) parent group targeting children’s challenging behavior; 18 parents were in a waitlist control group. All parents completed pre- and post-treatment measures on CPS learning targets, their child’s behavior, and relationship quality, and parents in the treatment group completed measures six months later. Only parents learning CPS reported significant improvements on learning targets and conflict, and changes were sustained six months later. Children’s behavior improved significantly in the treatment group by six-month follow-up. These findings support CPS parent groups as an economical and effective approach for parents of children with behavioral difficulties.

OBJECTIVES:This study aimed to evaluate the effectiveness of the Collaborative Problem Solving (CPS) approach in home-based family therapy and to explore two hypothesized mechanisms of change.METHOD:Sixty-seven families with children aged 3-12 years old completed a 12-week home-based CPS treatment program. Parent-report measures were completed pre- and post-intervention, including measures on parents' fidelity of using CPS, parents' empathy, children's executive functioning, children's behavioral difficulties, and parenting stress.RESULTS:There were significant reductions in children's behavioral difficulties and parenting stress, and significant improvements in children's executive functioning and parents' empathy. These improvements were greatest for parents who had the greatest fidelity to CPS. Improvements in children's executive functioning and parents' empathy mediated the relationship between parents' CPS fidelity and outcomes.CONCLUSIONS:These results provide evidence that home-based family treatment with CPS may achieve positive child and family outcomes by building children's executive function skills and improving parents' empathy.

In the last decade, many implementation frameworks have emerged that consolidate the research on implementation science, guiding purveyors and service agencies in improving implementation of evidence-based practices (EBPs). In this paper, we describe how the purveyor of one EBP utilized the active implementation frameworks (AIFs) to define and standardize strategies for site-wide implementation. We illustrate what implementation looked like before and after using AIFs to understand implementation, as well as some ways in which using the AIFs helped the purveyor identify, and then overcome, barriers to implementation. This paper provides a model for others who seek to use AIFs to guide their implementation practices, or more broadly, an illustration of how to use any implementation framework to ensure best practices in implementation.

This chapter describes various factors associated with evaluating the effects of implementing Collaborative Problem Solving (CPS). It first provides a summary of existing evidence regarding CPS outcomes and then demystifies the process of outcome evaluation for quality improvement by outlining the critical steps for examining outcomes in any clinical or educational setting. Finally, the chapter follows a fictional case example as it takes the necessary steps to complete an evaluation of outcomes and presents results to stakeholders.

Deficits in a range of skill domains (including executive functioning, emotion regulation, social cognition and language/communication) are associated with disrupted youth behavior and functioning across mental health diagnoses. The identification of skill deficits are important for effective treatment planning, particularly for personalized interventions. While there are multiple ways to assess these skills, parent/caregiver reports represent an important information source. To date, no single, brief measure has been developed that gathers parent/caregiver ratings across this range of constructs. We have developed a short caregiver-report questionnaire (the Thinking Skills Inventory; TSI), to screen for skill deficits. Here, we examine the reliability and validity of this rating scale in 384 youth who were consecutively referred for neuropsychiatric evaluation. A primary caregiver completed the TSI as well as other established measures. Exploratory and confirmatory factor analyses support five subscales on the TSI: Attention and Working Memory, Language and Communication, Emotion Regulation, Cognitive Flexibility, and Social Thinking Skills. The subscales showed moderate to high internal consistency (Cronbach’s alphas range from 0.84 to 0.91). Correlations with established caregiver-report measures confirm their convergent and discriminant validity, and associations with multiple clinical diagnoses and cross-diagnostic aggressive behavior further support the utility of the scale for our intended purpose. In sum, this free, brief measure is a valid and reliable way to identify variation in skill domains relevant to a range of psychopathology. The TSI may be useful in youth mental health settings to assist with treatment planning and to inform referral for further evaluation.