Children with dyslexia in second grade (7-8 years old) have difficulty remembering not only verbal information but also serial position information even for nonverbal items (Cowan, Hogan, Alt, Green, Cabbage, Brinkley, & Gray, 2017). We carried out a similar but more extensive study across elementary school Grades 1-4, including children in three diagnostic groups: 470 with typical development, 165 with dyslexia only, and 42 with both dyslexia and developmental language disorder (DLD). We examined standard spans and running spans for spoken digits, locations, and shapes. With increasing age, those with dyslexia alone caught up with children with typical development in at least one nonverbal task (standard location span). However, substantial memory differences between diagnostic groups remained in the higher grades and were most pronounced for children who have dyslexia and concomitant DLD (dyslexia+DLD), suggesting that DLD assessment is crucial. An interesting dissociation was observed in which diagnostic group differences for standard digit span emerged only in measures of item retention, whereas diagnostic group differences for other measures were largest for an index of the accuracy of the serial order of recalled items. Separate examination of two aspects of this dissociation, phonological item memory versus nonverbal serial order memory, showed individual differences on a continuum rather than discernable subgroups of children in any diagnostic group. Remediation of dyslexia should be designed to consider the importance of not only verbal working memory item capacity, but also serial position information for various types of information, which could impact text comprehension and other schoolwork.
Latent variable item response and structural equation models are widely used to model constructs and acknowledge measurement error in research settings and operational assessments. Such work often proceeds in stages, where the results of an analysis from an earlier stage are fed into the analysis at a later stage. However, common practices in single-stage and multistage estimation have weaknesses, including when viewed from a Bayesian perspective. This work extends recent developments in 2-Stage Bayesian approaches in structural equation modelling (SEM) to advance a general multistage approach where models are viewed as comprised of fragments that can be assembled in a modular way. The proposed approach is more in line with Bayesian principles and offers advantages over existing approaches. The approach is illustrated by applications in several different modelling scenarios, including: SEM for a wider class of situations than existing approaches have considered; and calibration and scoring situations encountered in operational assessment using item response theory (IRT). R functions for executing these analyses by interfacing with Mplus are provided and documented, as is code for running the examples.
To conduct rigorous evaluations of preventive interventions, it is foundational to establish the psychometric or measurement quality of the measures. Confirmatory factor analysis (CFA) is a popular method of modeling and evaluating measurement quality. Such analyses are typically conducted within a frequentist framework, which can pose challenges in an applied research setting due to the strong conditions required to establish measurement quality (e.g., sufficient sample size, exact measurement invariance (MI), and high sensitivity to group differences). An alternative set of approaches involves Bayesian methods, which offer several advantages. However, they remain underutilized by prevention scientists. The main goal of this paper is to illustrate several of the advantages that Bayesian methods offer in the context of analyses of data from the Alabama Parenting Questionnaire (APQ) through five examples. We illustrate the advantages of Bayesian methods over maximum likelihood in terms of result interpretation, and highlight how Bayesian methods allow us to express uncertainty in ways we always intended, avoiding misconceptions associated with frequentist approaches (Example 1). We also show how Bayesian methods help to avoid estimation problems (Example 2), examine parameter MI in both conventional and more flexible ways (Examples 3 and 4), and incorporate substantive prior information into our analysis (Example 5). By highlighting these advantages, we aim to motivate prevention researchers to consider using Bayesian methods for CFA and other analyses.
Research in education and behavioral sciences often involves the use of latent variable models that are related to indicators, as well as related to covariates or outcomes. Such models are subject to interpretational confounding, which occurs when fitting the model with covariates or outcomes alters the results for the measurement model. This has received attention in models for continuous observable variables but to date has not been examined in the context of discrete variables. This work demonstrates that interpretational confounding can occur in models for discrete variables, and develops a multistage Bayesian estimation approach to deal with this problem. The key features of this approach are that it is (a) measurement preserving, in that it precludes the possibility of interpretational confounding, and (b) uncertainty preserving, in that the uncertainty from the earlier stage of estimating the measurement model is propagated to the second stage of estimating the relations between the latent variable(s) and any covariates or outcomes. Previous work on these methods had only considered models for continuous observed variables, and software was limited to models with a single latent variable and either covariates or outcomes. This work extends the approach and software to a more general class of solutions, including discrete variables, illustrating the procedures with analyses of real data. Functions for conducting the analyses in widely available software are provided.
Working memory encompasses the limited incoming information that can be held in mind for cognitive processing. To date, we have little information on the effects of bilingualism on working memory because, absent evidence, working memory tasks cannot be assumed to measure the same constructs across language groups. To garner evidence regarding the measurement equivalence in Spanish and English, we examined second-grade children with typical development, including 80 bilingual Spanish-English speakers and 167 monolingual English speakers in the United States, using a test battery for which structural equation models have been tested - the Comprehensive Assessment Battery for Children - Working Memory (CABC-WM). Results established measurement invariance across groups up to the level of scalar invariance.
Parents' management of their children's sibling relationships, or sibling-focused parenting, has substantial theoretical and practical importance but is rarely studied. This study's goals were to describe dimensions of sibling-focused parenting and to examine sociocultural resources and challenges as potential correlates among Latinx mothers and fathers in 262 families with two children in middle childhood. Families were recruited from 11 public elementary schools, and caregivers (248 mother figures; 118 father figures) participated in a home visit and phone interviews at the onset of the study. Sibling-focused parenting included three dimensions: positive guidance (10 items), nonintervention (four items), and authoritarian control (five items). Parents rated positive guidance as their most frequent strategy, and comparisons of mothers and fathers from the same families revealed that mothers engaged in more sibling-focused parenting overall than fathers. Regarding correlates, mothers' familism values and mothers' and fathers' family cohesion reports were associated with more positive guidance and mothers' cohesion was negatively related to nonintervention in sibling conflicts. For mothers only, parenting stress was linked to all three dimensions of sibling-focused parenting-negatively to guidance and positively to authoritarian control and nonintervention; maternal depressive symptoms were positively linked to authoritarian control. Economic hardship was not a significant correlate of any dimension. Findings suggest that sibling-focused parenting is a key domain of parenting in need of further research. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Missing data are a common occurrence in analyses of multivariate data, including in multilevel modeling. Bayesian approaches to handling missing data in multilevel modeling have garnered increasing attention, either on their own or in service of multiple imputation. However, these applications are largely confined to specific models or missingness patterns. The current work provides a coherent account of Bayesian analysis of multilevel models in the presence of missing data on the outcomes, level-1 predictors, and level-2 predictors, that covers the main aspects of the models and missingness. In doing so, this work provides a grounding for estimation in fully Bayesian approaches that employ Gibbs sampling, and provides an account of how to generate the imputations in the first phase of a multiple imputation approach.
In latent variable models, interpretational confounding occurs when the inclusion of a covariate or outcome when fitting the model alters the results for the measurement model. Commonly used estimation procedures do not preclude this possibility. Multi-stage estimation approaches preclude interpretational confounding, but most are limited in that they do not properly propagate uncertainty from earlier stages to later stages. This work introduces a measurement and uncertainty preserving approach to factor analytic models with covariates or outcomes, which additionally supports procedures for conducting diagnostic model-data fit analyses. These are examined in simulation studies, where they perform favorably relative to existing strategies, and illustrated with analyses of real data. Functions for conducting the analyses in freely available software are provided.
Although there is recognition that there may be differential outcomes for groups of students within schools, examination of outcomes for subgroups presents challenges to researchers and policymakers. It complicates analytic procedures, particularly when the number of students per school in the subgroup is small. We explored five alternatives for applying a three-level multilevel growth modeling framework to examine school-level achievement for a select subgroup of students (students with disabilities) using a large longitudinal database tracking reading achievement. The alternatives vary in terms of use of subgroup only or all student data, use of student-level predictors, and method of linking student or school-level outcomes to school effectiveness indices. Correlations from .57 to .99 among alternatives suggest the choice of how to derive school-level outcomes for a subgroup has consequences for inferences about the school's effectiveness with the subgroup. Researchers' assumptions and data available should guide the selection of an approach.
Meta-analyses on the relation between socioeconomic status (SES) and performance on measures of cognitive ability and achievement arrive at the same general conclusion of a small to medium association. Advancements in methods make possible for meta-analyses to examine specific pathways linking SES to cognitive ability and achievement, as well as the moderators of these pathways. In this study, we conducted a systematic overview of meta-analyses on SES to address three research questions: 1) what is the direction and overall strength of association between SES and performance on measures of cognitive ability and achievement, and how precise are the effect sizes reported? 2) to what extent have meta-analyses examined moderation by components of SES, age, sex, and race/ethnicity? and 3) to what extent have meta-analyses examined mechanisms linking SES to cognitive ability and achievement? We conducted a systematic search using online archives (i.e., PsycINFO, ERIC, PubMed, Sociological Abstracts, and Web of Science), searching issues in Psychological Bulletin and Review of Educational Research, and examining references and citations. We identified 14 meta-analyses published between 1982 and 2019. These meta-analyses consistently reported positive associations of small to medium magnitude, indicating that SES is a meaningful contributor to the development of cognitive ability and achievement. Fewer meta-analyses reported evidence of moderation by age, sex, and race/ethnicity. None of the meta-analyses directly examined mechanisms, but provided evidence of possible mechanisms for future research. We suggest that meta-analyses can increase their contribution to future research, interventions, and policy by narrowing their focus on specific pathways.
Obtaining values for latent variables in factor analysis models, also referred to as factor scores, has long been of interest to researchers. However, many treatments of factor analysis do not focus on inference about the latent variables, and even fewer do so from a Bayesian perspective. Researchers may therefore be ill-acquainted with Bayesian thinking on this issue, despite the fact that certain existing procedures may be seen as Bayesian to some extent. The focus of this paper is to provide a conceptual grounding for Bayesian inference for latent variables, articulating not only what Bayesian inference has to say about values for latent variables, but why Bayesian inference is suited for this problem. As to why, it is argued that the notion of exchangeability motivates the form of factor analysis, as well as Bayesian inference for latent variables. The argument is supported by documenting the widespread use of Bayesian inference in analogous settings, including latent variables in other measurement models, multilevel models, and missing data. As to what, this work describes a Bayesian analysis when other parameters are known, as well as partially and fully Bayesian analyses when other parameters are unknown. This facilitates a discussion of various choices researchers have when adopting Bayesian approaches to inference about latent variables.
Purpose: The purpose of this study was to use an established model of working memory in children to predict an established model of word learning to determine whether working memory explained word learning variance over and above the contributions of expressive vocabulary and nonverbal IQ. Method: One hundred sixty-seven English-speaking second graders (7- to 8-year-olds) with typical development from two states participated. They completed a comprehensive battery of working memory assessments and six word learning tasks that assessed the creation, storage, retrieval, and production of phonological and semantic representations of novel nouns and verbs and the ability to link those representations. Results: A structural equation model with expressive vocabulary, nonverbal IQ, and three working memory factors predicting two word learning factors fit the data well. When working memory factors were entered as predictors after expressive vocabulary and nonverbal IQ, they explained 45% of the variance in the phonological word learning factor and 17% of the variance in the semantic word learning factor. Thus, working memory explained a significant amount of word learning variance over and above expressive vocabulary and nonverbal IQ. Conclusion: Results show that working memory is a significant predictor of dynamic word learning over and above the contributions of expressive vocabulary and nonverbal IQ, suggesting that a comprehensive working memory assessment has the potential to identify sources of word learning difficulties and to tailor word learning interventions to a child's working memory strengths and weaknesses. Supplemental Material: https://doi.org/10.23641/asha.19125911
Introduction: The association between socioeconomic status (SES) and depressive symptoms is well documented, yet less attention has been paid to the methodological factors contributing to between-study variability. We examined the moderating role of range restriction and the depressive-symptom measurement instrument used in estimating the correlation between components of SES and depressive symptoms. Methods: We conducted an individual participant data meta-analysis of nationally-representative, public-access datasets in the United States. We identified 123 individual datasets with a total of 1,655,991 participants (56.8 % female, mean age = 40.33). Results: The presence of range restriction was associated with larger correlations between income and depressive symptoms and with smaller correlations between years of education and depressive symptoms. The measurement instrument of depressive symptoms moderated the association for income, years of education, and occupational status/prestige. The Center for Epidemiological Studies-Depression scale consistently produced larger correlations. Higher measurement reliability was also associated with larger correlations. Limitations: This study was not a comprehensive review of all measurement instruments of depressive symptoms, focused on datasets from the United States, and did not examine the moderating role of sample characteristics. Discussion: Methodological characteristics, including range restriction of SES and instrument of depressive symptoms, meaningfully influence the observed magnitude of association between SES and depressive symptoms. Clinicians and researchers designing future studies should consider which instrument of depressive symptoms is suitable for their purpose and population.
Detection and responding to a player’s affect are important for serious games. A method for this purpose was tested within Chem-o-crypt, a game that teaches chemical equation balancing. The game automatically detects boredom, flow, and frustration using the Affdex SDK from Affectiva. The sensed affective state is then used to adapt the game play in an attempt to engage the player in the game. A randomized controlled experiment incorporating a Dynamic Bayesian Network that compared results from groups with the affect-sensitive states vs those without revealed that measuring affect and adapting the game improved learning for low domain-knowledge participants.
Dynamic Bayesian networks (DBNs; Reye, 2004) are a promising tool for modeling student proficiency under rich measurement scenarios (Reichenberg, 2018). These scenarios often present assessment conditions far more complex than what is seen with more traditional assessments and require assessment arguments and psychometric models capable of integrating those complexities. Unfortunately, DBNs remain understudied and their psychometric properties relatively unknown. The current work aimed at exploring the properties of DBNs under a variety of realistic psychometric conditions. A Monte Carlo simulation study was conducted in order to evaluate parameter recovery for DBNs using maximum likelihood estimation. Manipulated factors included sample size, measurement quality, test length, the number of measurement occasions. Results suggested that measurement quality has the most prominent impact on estimation quality with more distinct performance categories yielding better estimation. From a practical perspective, parameter recovery appeared to be sufficient with samples as low as N = 400 as long as measurement quality was not poor and at least three items were present at each measurement occasion. Tests consisting of only a single item required exceptional measurement quality in order to adequately recover model parameters.
The primary purpose of this study was to compare the working memory performance of monolingual English-speaking second(-)grade children with dyslexia (N = 82) to second-grade children with typical development (N = 167). Prior to making group comparisons, it is important to demonstrate invariance between working memory models in both groups or between-group comparisons would not be valid. Thus, we completed invariance testing using a model of working memory that had been validated for children with typical development (Gray et al., 2017) to see if it was valid for children with dyslexia. We tested three types of invariance: configural (does the model test the same constructs?), metric (are the factor loadings equivalent?), and scalar (are the item intercepts the same?). Group comparisons favoured the children with typical development across all three working memory factors. However, differences in the Focus-of-Attention/Visuospatial factor could be explained by group differences in non-verbal intelligence and language skills. In contrast, differences in the Phonological and Central Executive working memory factors remained, even after accounting for non-verbal intelligence and language. Results highlight the need for researchers and educators to attend not only to the phonological aspects of working memory in children with dyslexia, but also to central executive function.
Socioeconomic status (SES) is a widely researched construct in developmental science, yet less is known concerning relations between SES and adaptive behavior. Specifically, is the relation linear, with higher SES associated with better outcomes, or does the direction of association change at different levels of SES? Our aim was to examine linear ("more is better") and quadratic ("better near the middle") associations between components of SES (i.e., income, years of education, occupational status/prestige) and depressive symptoms (Center for Epidemiologic Studies-Depression Scale), and to explore moderation by developmental period (adolescence, young, middle, and older adulthood), gender/sex (female, male), and race/ethnicity (Asian American, Black, Latinx, multiracial, Native American, White). We hypothesized that there would be more support for a model containing quadratic associations. We conducted a two-stage meta-analytic structural equation model of 60 data sets (27,242 correlations, 498,179 participants) within the United States, accounting for dependencies between correlations, which were identified via the Interuniversity Consortium for Political and Social Research and handled using a two-step approach. Income was quadratically associated with depressive symptoms, but the quadratic model did not explain more variance in depressive symptoms than the linear model. Developmental period and race/ethnicity moderated the associations: Income was quadratically associated with depressive symptoms among middle-aged adults, and years of education were quadratically associated with depressive symptoms among White samples. Our findings suggest that researchers and clinical practitioners should consider the elevated risk of depressive symptoms for individuals from low and high-income backgrounds in the United States. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
This chapter characterizes several principles of assessment and how they play out in simulation-based assessment environments. It is argued that these may be (optimistically) viewed as opportunities, as well as (considerably less optimistically) viewed as challenges to assessment, and in particular to psychometrics. The discussion is framed through an evidence-centered design (ECD) perspective (Mislevy et al., 2003), and trades heavily on a notion of task design in terms of the problem space, tool, space, solutions space, and response space (Behrens et al., 2012a). Examples of simulation-based assessments are used throughout to illustrate the key ideas.
Use of Bayesian methods has proliferated in recent years as technological and software developments have made Bayesian methods more approachable for researchers working with empirical data. Connected with the increased usage of Bayesian methods in empirical studies is a corresponding increase in recommendations and best practices for Bayesian methods. However, given the extensive scope of Bayes, theorem, there are various compelling perspectives one could adopt for its application. This paper first describes five different perspectives, including examples of different methodologies that are aligned within these perspectives. We then discuss how the different perspectives can have implications for modeling and reporting practices, such that approaches and recommendations that are perfectly reasonable under one perspective might be unreasonable when viewed from another perspective. The ultimate goal is to show the heterogeneity of defensible practices in Bayesian methods and to foster a greater appreciation for the variety of orientations that exist. (PsycInfo Database Record (c) 2023 APA, all rights reserved).