With the growing emphasis on accountability in mental health services, outcomes management strategies are gaining popularity. However, for these techniques to be credible, it is necessary to ensure the reliability of clinical data. In other words, outcomes measures must accurately reflect the actual status of service recipients. This article presents the use of the measurement audit as one means of monitoring and improving the reliability of outcomes measurements. The methods and findings from an audit of crisis assessment workers for children in state custody are presented. Clinical assessments completed at the time of service were compared with assessments using the same measure completed via retrospective file review. Findings suggest generally good reliability, 0.72 overall, with some variation by provider and type of information.

OBJECTIVE To examine differences in risk levels of psychiatric hospital referrals received from residential treatment centers (RTCs) as a measure of service quality. METHOD This prospective study used the Childhood Severity of Psychiatric Illness (CSPI), a reliable measure of psychiatric severity and factors thought to affect decision-making. Psychiatric referrals were wards of the state from the 10 largest RTCs in one metropolitan county and were screened by mobile crisis workers for admission appropriateness. After interviewing clients, the independent crisis workers completed the CSPI. RESULTS Referrals were placed into 1 of 2 categories based on CSPI ratings: high-risk or low-risk. Referrals were deemed high-risk if they displayed moderate or severe levels of suicidality, dangerousness to others, or psychotic symptoms on the CSPI. Results showed statistically significant variation in the level of risk of referrals received from the 10 RTCs. In addition, RTCs that had high rates of low-risk referrals were rated by crisis workers as demonstrating poorer supervision of their clients. CONCLUSIONS The residential treatment providers in our sample are expected to provide intensive treatment to children and adolescents with serious emotional and behavioral problems. Our finding that some RTCs are making low-risk referrals suggests that they are struggling to meet the needs of some of their clients. Our findings can be used to inform quality improvement efforts at RTCs that are currently struggling.

OBJECTIVE:To examine factors related to psychiatric hospitalization decision and length of stay of wards of the illinois Department of Child and Family Services.METHOD:A prospective design was implemented using the Childhood Severity of Psychiatric Illness (CSPI), a reliable, quantitative measure of psychiatric severity and its mediating factors. The CSPI was completed by hospital screeners upon conclusion of their crisis interviews. In addition to completing the CSPI, workers reported on demographic information, DSM-IV diagnoses, prescreening living arrangements, and length of hospital stay.RESULTS:CSPI variables could effectively predict decision to admit versus deflect. The overall accuracy of this statistically significant prediction model was 77.9%, which was replicated on a new sample. Factors associated with decision to hospitalize are clinical in nature; ratings of suicidality, dangerousness, and impulsivity contributed the most to the model. Predicting length of stay was only moderately successful. Despite achieving significance, the model accounted for just 15.1% of length of stay variance using a multiple regression. Factors associated with length of stay were largely nonclinical in nature: living arrangement stability, region of the hospitalization, and age.CONCLUSIONS:These results can be used to assess how decisions regarding level and duration of care are currently being made as a point of departure for quality improvement efforts.