Youth were experiencing mental health crises before the onset of the COVID-19 pandemic. Following this onset, their needs for mental health services have only increased. Yet, researchers encounter barriers to confronting these crises. The effects of therapies tested in controlled trials in the present day appear to be no more potent than those of their predecessors tested in trials conducted decades ago. Across these decades of scholarly work, researchers have invested far more of their efforts toward improving technologies for therapies than they have toward improving technologies for the assessment tools used to estimate therapeutic effects. The tools used today look a lot like those used in the 1970s—mainly surveys and interviews—and our strategies for integrating the data these tools produce focus on the sliver of their data that converge or yield the same results about youth mental health. Decades of work reveal that these integration strategies are incompatible with the data conditions that typify youth mental health assessments. We must invest in innovative assessment tools and integration strategies that capitalize on all the valid data produced by these conditions. This paper details pioneering directions in future research about psychological assessment. We describe the conceptual foundations underlying these research directions and highlight recent work by the authors and others supporting this pursuit. If we empower the assessment researchers of today to develop technologically innovative assessment tools and integration strategies, then we equip the therapy researchers of tomorrow to demonstrate that investing in therapy technologies pays off.
Adults (e.g., caregivers, teachers) play a key role in youth mental health services, even as the perspectives of youth themselves are also critical to intervention success. Prior studies indicate that caregivers and youth frequently disagree on the rationale for the interventions youth receive (i.e., needs for intervention) and the plan for achieving intervention success (i.e., goals for intervention). Yet, all previous work is based on clinic samples. None have included the perspectives of teachers. We developed the Kids' Behavior in Context Scales (KICS) to assess the contexts in which youth needs and goals manifest, which requires psychometrically sound procedures for detecting needs and goals. A school-based sample of 173 sixth- to eighth-grade youth, their caregivers, and their teachers each identified needs for intervention (e.g., aggression, anxiety, inattention), as well as goals for intervention (e.g., controlling impulses, building healthy relationships, relaxation). We observed low levels of between-informant agreement on needs and goals for intervention, with kappas ranging from .01 to .11. For only 2% of youth, all three informants endorsed the same pair of needs and goals. The KICS reveals that informant disagreements occur more frequently in school-based assessments relative to other service environments (e.g., hospitals, community mental health clinics).
Tailoring psychosocial interventions requires linking youth mental health concerns to the social contexts in which they manifest. We designed the Kids' Behavior in Context Scales (KICS) to gather data about these social contexts & horbar;experiences at school, home, and/or with peers, for example & horbar;and their links to youth mental health services. A school-based sample of 173 sixth- to eighth-grade youth, their caregivers, and their teachers each rated social contexts connected to needs for intervention (e.g., aggression, anxiety, inattention) and goals for intervention (e.g., building healthy relationships, relaxation). On the KICS, higher ratings indicate higher contextual stability of youth needs and goals. For each informant, their ratings demonstrated criterion-related validity connected to scores taken from well-established measures of youth psychosocial functioning. Cross-informant agreement on social context ratings was low-to-moderate for needs (average r = .24) and near-zero for goals (average r = .01). These patterns resulted in interpretable structures for discrepant results among informants' social context ratings for needs but not for goals, and thus, distinct correlations with validity criteria when social context ratings were integrated. Social contexts linked to needs for intervention may be more contextually stable than those related to goals for intervention. The KICS opens doors to using contextual data when tailoring youth mental health services.
We conducted a conceptual replication of Pigott et al.’s study of outcome-reporting bias, wherein they compared intervention outcomes reported in unpublished education dissertations with corresponding published versions. For our replication, we identified a sample of 40 special education dissertations with matched journal publications and found that statistically significant intervention outcomes from dissertations were 1.48 times more likely to be published compared with nonsignificant outcomes. Significant moderators of this effect included type of intervention outcome (academic), type of research design (randomized controlled trial), participant race (with samples greater than or equal to 50% non-White), and type of disability/exceptionality (high incidence). We found that few dissertation authors published their work, providing further evidence for the much-needed inclusion of dissertations in systematic reviews.
Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We advance EBA in three sections in this article. First, we describe an empirically grounded framework, the Operations Triad Model (OTM), to inform EBA decision-making in the articulation of relevant educational theory. Originally designed for interpreting mental health assessments, we describe features of the OTM that facilitate its fusion with educational theory, namely its falsifiability. In turn, we cite evidence to support the OTM's ability to inform hypothesis generation and testing, study design, instrument selection, and measurement validation. Second, we describe quality indicators for interpreting psychometric data about measurement tools, which informs both the development and selection of measures and the process of measurement validation. Third, we apply the OTM and EBA to research in special education in two contexts: (a) empirical research for causal explanation and (b) implementation science research. We provide open data resources to advance measurement validation and conclude with future directions for research.
The Council for Exceptional Children’s Division for Research (CEC-DR) is pleased to introduce a special issue designed to update and advance quality indicators for research in special education, advancing work originally published in a 2005 special issue of Exceptional Children (Volume 71, Issue 2). Then, as now, special education research has been characterized by a “long and cherished tradition” of diverse research methods (Graham, 2005, p. 135). Research methods in each area have advanced considerably since 2005. Thus, the articles in this 2023 special issue provide guidance for researchers who are conducting research that will continue to move the field of special education forward. The issue includes articles focused on quality indicators for group-design research, single-case-design research, secondary data analysis, systematic literature reviews, mixed-methods research (MMR), qualitative research, and evidencebased assessment. Moreover, these seven articles foreshadow how research methods will continue to evolve over the next decade, particularly in deepening understanding of equity and open-science practices. First in the issue is an article focused on quality indicators for group-design research. Jessica Toste, Jessica Logan, Karrie Shogren, and Brian Boyd provide an expanded set of indicators designed to advance knowledge about for whom and under what conditions interventions, programs, and practices are more or less effective for students with disabilities. The authors introduce new quality indicators to guide decisions related to the design, implementation, and analysis of research of groups of people in special education. Next, to advance quality indicators for single-case-design research, Jennifer Ledford, Joseph Lambert, James Pustejovsky, Nicole Hollins, and Erin Barton extend previous standards by providing guiding principles and recommendations that advance rigor in internal validity, generality and acceptability, and reporting. They also promote considerations for single-case synthesis, which has grown substantially since the 2005 quality indicators were published, and discuss how the field can assess accumulated evidence for certain practices or intervention approaches. The topic of the third article is new to this special issue. It introduces a series of recommendations for secondary data analysis. Allison Lombardi, Graham Rifenbark, and Ashley Taconet highlight preregistration as a tool for researchers to share innovative questions and analytic approaches as well as increase transparency. To that end, they describe quality indicators for secondary data analysis using applied examples from published studies based on two iterations of the National Longitudinal Transition Study (NLTS2 and NLTS2012). Michelle Cumming, Elizabeth Bettini, and Jason Chow propose four core principles to guide scholars in conducting high-quality systematic literature reviews: coherence, contextualization, generativity, and transparency. The authors described the application of these principles to each stage of the review process and best practices for enhancing the rigor, relevance, and credibility of meta-analyses, systematic narrative reviews, and qualitativemeta-syntheses. They also describe considerations for advancing equity and connecting research findings to practice and policy. The fifth article, a topic also new to this special issue, focuses on best practices in Editorial
Divided into two volumes, Handbook of Special Education Research provides a comprehensive overview of critical issues in special education research. This first volume addresses key topics in theory, methods, and development, exploring how these three domains interconnect to build effective special education research. Each chapter features considerations for future research and implications for fostering continuous improvement and innovation. Essential reading for researchers and students of special education, this handbook brings together diverse and complementary perspectives to help move the field forward. This chapter outlines theory, development, and application of behavior analysis as it relates to special education research and practice. One particular branch of behavior analysis, applied behavior analysis (ABA), is embedded in both the history and practice of special education. ABA is a scientific approach to understanding and changing behavior in ways that address problems of social importance (Wolf, 1978). Using a variety of research-based strategies, ABA practitioners and behaviorally oriented educators dedicate their practice to the systematic application of instructional technologies that develop the skills (e.g., academic, behavioral, daily living) most likely to increase quality of life in individuals with and without disabilities. Cognitive science includes cognitive psychology, developmental cognitive psychology, and educational and cognitive neuroscience (see Chapters 7 and 18). In this chapter, we illustrate the contribution of cognitive science to understanding the nature of learning disabilities (LDs) and to intervention design. We discuss historical and current influences of cognitive science, including how (a) academic interventions for LDs were motivated by schema theory and theories of metacognition and metacognitive development; (b) cognitive principles of learning have been applied to academic interventions; and (c) models of reading and mathematical cognition inform understanding of and interventions for dyslexia, reading comprehension disabilities, and math disabilities. We also present some directions for future research.
Research indicating many study results do not replicate has raised questions about the credibility of science and prompted concerns about a potential reproducibility crisis. Moreover, most published research is not freely accessible, which limits the potential impact of science. Open science, which aims to make the research process more open and reproducible, has been proposed as one approach to increase the credibility and impact of scientific research. Although relatively little attention has been paid to open science in relation to single-case design, we propose that open-science practices can be applied to enhance the credibility and impact of single-case design research. In this article, we discuss how open-science practices align with other recent developments in single-case design research, describe four prominent open-science practices (i.e., preregistration, registered reports, data and materials sharing, and open access), and discuss potential benefits and limitations of each practice for single-case design.
Over 60 years of research reveal that informants who observe youth in clinically relevant contexts (e.g., home, school)-typically parents, teachers, and youth clients themselves-often hold discrepant views about that client's needs for mental health services (i.e., informant discrepancies). The last 10 years of research reveal that these discrepancies reflect the reality that (a) youth clients' needs may vary within and across contexts and (b) informants may vary in their expertise for observing youth clients within specific contexts. Accordingly, collecting and interpreting multi-informant data comprise "best practices" in research and clinical care. Yet, professionals across settings (e.g., health, mental health, school) vary in their use of multi-informant data. Specifically, professionals differ in how or to what degree they leverage multi-informant data to determine the goals of services designed to meet youth clients' needs. Further, even when professionals have access to multiple informants' reports, their clinical decisions often signal reliance on one informant's report, thereby omitting reports from other informants. Together, these issues highlight an understudied research-to-practice gap that limits the quality of services for youth. We advance a framework-the Needs-to-Goals Gap-to characterize the role of informant discrepancies in identifying youth clients' needs and the goals of services to meet those needs. This framework connects the utility of multi-informant data with the reality that services often target an array of needs within and across contexts, and that making decisions without accurately integrating multiple informants' reports may result in suboptimal care. We review evidence supporting the framework and outline directions for future research.
Valid and reliable teacher ratings serve as the foundation for screening and assessment of youth with behavioral disorders and twin studies offer an opportunity to study those ratings. We conducted a meta-analysis of 15 empirical investigations of aggressive and rule-breaking behavior using teacher ratings in the context of a twin research design. We retrieved n = 53 correlations from n = 7,885 pairs of monozygotic (MZ) twins and n = 67 correlations from n = 11,696 pairs of dizygotic (DZ) twins with two goals: (a) to test the best-fitting models for predicting similarity (correlations) in teacher ratings of MZ and DZ twin pairs and (b) to identify significant predictors in the respective models. We found that, for both MZ and DZ twins, the best-fitting model included all predictors and interactions. In the MZ model, we found a significant positive interaction between the percentage of same teachers who completed the ratings and male twins; in the DZ model, we found a significant negative interaction between the percentage of same teachers who completed the ratings and opposite sex twins. Teacher ratings converged and diverged in expected ways, advancing research in the context of the attributions-bias-context (ABC) model of informant ratings.
Attention-deficit hyperactivity disorder (ADHD) is among the most commonly diagnosed disorders of children and youth. Young people receive their ADHD diagnoses and medical treatment in primary health care settings and can experience a range of behavioral and educational disabilities treated in the clinic, at home, and at school. We propose a team-based collaborative care model (TBCCM) to foster communication and collaboration among health care and education teams, embedding implementation science methods to promote and sustain evidence-based practices for youth with ADHD. Key features of the model include (a) effective leadership and teamwork within the two universal systems of education and health care, (b) use of data from multiple informants who describe and monitor student behavior within and across contexts, and (c) adoption and adaptation of evidence-based practices. We expect that these efforts to embed implementation science methods within a collaborative team structure will improve the uptake of evidence by intervention teams in the two systems, and thus optimize outcomes for children and youth with ADHD.
Secondary students with high incidence disabilities who also display disruptive behaviors struggle to be successful in general education settings. As a result, general education teachers are looking for ways to utilize technology to provide them with opportunities to implement evidence-based interventions in their classrooms. In this study, teachers used MoBeGo, an iPad application, in a single-case withdrawal design (ABAB), to implement self-monitoring in high school general education classrooms with four students who received special education services for a high incidence disability. The results of this study indicate that teachers could implement MoBeGo with fidelity to improve students' academic engagement and appropriate behavior. Additionally, both the teachers and students rated MoBeGo as a socially valid intervention. Implications for practice and future research are discussed.
There is general agreement in the research literature that youth in juvenile justice facilities are more likely to experience mental health disorders than their general population peers. The purpose of this systematic review and meta-analysis was to evaluate the methodological characteristics and effectiveness of mental health interventions delivered in juvenile justice settings on symptoms associated with internalizing disorders. The 11 studies included in the current review incorporated pretest-posttest research designs and were conducted with juveniles in secure facilities that reported outcome measures of depression, anxiety, posttraumatic stress disorder, or internalizing disorders. Meta-analytic findings indicate mixed results for interventions affecting internalizing symptoms and varying results between studies implementing an experimental design compared to those using a single group non-experimental design. Additionally, no studies examined how interventions could be incorporated into daily activities in juvenile justice facilities, such as school and classroom activities. Lastly, the limited number of studies included in the current review indicates a continued need for further experimental research on the effectiveness of mental health interventions delivered to youth in juvenile justice facilities.
In this manuscript, we (a) briefly describe proposed open-science practices to increase transparency of research in special education and related disciplines, and (b) provide recommendations for research funders, professional societies, journal editors and publishers, and individual researchers to support awareness, exploration, and adoption of open science.