Bayesian diagnostic classification models (Bayesian DCMs) are effective for diagnosing students' skills. Research on the evaluation of relative model fit indices for DCMs using Bayesian estimation, however, is deficient. This study introduces the performance of Bayesian relative model fit indices, the widely applicable information criterion (WAIC) and leave-one-out cross-validation using Pareto-smoothed importance sampling (PSIS-LOO), in comparison to simpler and more widely used deviance information criterion (DIC). The simulation study evaluates the performance of WAIC and PSIS-LOO by detecting the true model with varying sample sizes, item qualities, and prior information levels. The results of the study indicate that WAIC and PSIS-LOO primarily favored the generating model; however, occasional inconsistencies were observed. This study recommends using WAIC and PSIS-LOO when the data is assumed to follow a simpler model and the models are estimated under uninformative priors, and DIC when the data is assumed to follow a more complex model.
Despite their positive relationships with health outcomes, few studies directly assess the relationships among religiosity and hope. Using item factor analysis (N = 630) within a religiously diverse United States sample, we hypothesized fundamentalism (H1: Intratextual Fundamentalism Scale, ITFS; Multidimensional Fundamentalism Inventory, MDFI; and Religious Fundamentalism Scale, RFS) and hope measures (H2: Adult Hope Scale, AHS; Integrative Hope Scale, IHS) would demonstrate acceptable psychometrics and statistically (H3) and practically significant relationships (H4). The ITFS possessed near perfect psychometrics (CFI = 1.000, TLI = 1.000, RMSEA = 0.000, SRMR = 0.009, omega = 0.92), but other measures needed modifications. After Bonferroni corrections, we found statistically significant relationships among the RFS and the AHS (r = .16) and IHS (r = .155) as well as the ITFS and the AHS (r = .155) and IHS (r = .162), all at a small effect size. However, there were no statistically significant relationships among the MDFI and the hope measures (H3). No association among a fundamentalism measure and a hope measure reached practical significance (H4). Given these results, we computed correlations among the original scales and found similar results among hope and fundamentalism measures (r = − .002–0.152). These findings indicate direct relationships among measures of hope and fundamentalism, but no relationship met the criteria for practical significance—before or after scale modifications. We discuss the implications of these findings regarding the integration of religiosity and spirituality into mental health research, training, and practice.
Objective. The health effects of SAPHO (synovitis, acne, pustulosis, hyperostosis, and osteitis) syndrome and chronic nonbacterial osteomyelitis (CNO) have not been well studied. We assessed health-related quality of life (HRQOL) in adults with SAPHO-CNO and performed a review of EQ-5D questionnaire outcomes among similar chronic rheumatic and inflammatory diseases. Methods. We enrolled patients in the first US-based SAPHO-CNO prospective registry and assessed their HRQOL using the EQ-5D index and EQ-5D visual analog scale (VAS). A focused scoping review was performed of EQ-5D-related outcomes among disease cohorts with phenotypic similarities to SAPHO-CNO. Results. In the 138 participants with SAPHO-CNO, mean age was 37.7 (SD 15.0) years with a mean disease duration of 6.4 (SD 6.2) years. All subjects had musculoskeletal involvement (with 43% having spine involvement) and 46% had skin involvement. Mean 5-level EQ-5D index value was 0.75 (SD 0.16) and mean EQ-VAS was 63.0 (SD 21.0). Most patients were affected by pain/discomfort (86%), difficulty in carrying out usual activities (65%), and anxiety/depression (54%); over one-third reported problems in all5 EQ-5D domains. These adults with SAPHO-CNO had similarly reduced EQ-5D and VAS values, and similar high proportion of perceived problems, as in other inflammatory disease cohorts. Conclusion. The HRQOL among adults with SAPHO-CNO is low, with more than half of patients experiencing greater impact on perceived pain, difficulty performing usual activities, and feelings of anxiety or depression. Future therapies should address the multidimensional nature of SAPHO-CNO, targeting HRQOL as a major outcome.
In classroom assessments, examinees can often answer test items multiple times, resulting in sequential multiple-attempt data. Sequential diagnostic classification models (DCMs) have been developed for such data. As student learning processes may be aligned with a hierarchy of measured traits, this study aimed to develop a sequential hierarchical DCM (sequential HDCM), which combines a sequential DCM with the HDCM, and investigate classification accuracy of the model in the presence of hierarchies when multiple attempts are allowed in dynamic assessment. We investigated the model's impact on classification accuracy when hierarchical structures are correctly specified, misspecified, or overspecified. The results indicate that (1) a sequential HDCM accurately classified students as masters and nonmasters when the data had a hierarchical structure; (2) a sequential HDCM produced similar or slightly higher classification accuracy than nonhierarchical sequential LCDM when the data had hierarchical structures; and (3) the misspecification of the hierarchical structure of the data resulted in lower classification accuracy when the misspecified model had fewer attribute profiles than the true model. We discuss limitations and make recommendations on using the proposed model in practice. This study provides practitioners with information about the possibilities for psychometric modeling of dynamic classroom assessment data.
Prior research on the Community of Inquiry (CoI) framework has a limited amount of work which uses structural techniques to confirm the factorial structure of the CoI. The current study investigates the structural relationships among the three elements of the CoI framework (cognitive presence, teaching presence, and social presence), but extends the prior literature by testing the inclusion of a learning presence factor as well as a unifying higher-order online educational presence factor. Using a hierarchical confirmatory factor analysis (CFA) with data collected from an online survey of 709 students enrolled in online courses across the U.S, we investigated (a) the relationships between online educational experience as a higher-order factor and the three original CoI elements as lower-order factors and (b) conducted a hierarchical model to investigate the relationships and model fit indices with the additional learning presence CoI element. The results of the models suggested adequate fit for a model with a higher-order construct, supporting a model which provides a more comprehensive picture of the CoI constructs. Furthermore, while the higher order online learner’s educational experience most strongly influenced the teaching presence subfactor, it also displayed significant impacts on the other factors. Finally, the addition of a learning presence construct did not decrease model fit and added theoretical depth to the CoI model. The data support the use of a multi-level revised CoI model in future research to better understand online educational success, and suggests that practitioners should develop approaches to enhancing learning presence in online educational settings.
Recently, Bayesian diagnostic classification modeling has been becoming popular in health psychology, education, and sociology. Typically information criteria are used for model selection when researchers want to choose the best model among alternative models. In Bayesian estimation, posterior predictive checking is a flexible Bayesian model evaluation tool, which allows researchers to detect Q-matrix misspecification. However, model selection methods using posterior predictive checking (PPC) for Bayesian DCM are not well investigated. Thus, this research aims to propose a novel model selection approach using posterior predictive checking with limited-information statistics for selecting the correct Q-matrix. A simulation study was conducted to examine the performance of the proposed method. Furthermore, an empirical example was provided to illustrate how it can be used in real scenarios.
This paper demonstrates the process of invariance testing in diagnostic classification models in the presence of attribute hierarchies via an extension of the log-linear cognitive diagnosis model (LCDM). This extension allows researchers to test for measurement (item) invariance as well as attribute (structural) invariance simultaneously in a single analysis. The structural model of the LCDM was parameterized as a Bayesian network, which allows attribute hierarchies to be modeled and tested for attribute invariance via a series of latent regression models. We illustrate the steps for carrying out the invariance analyses through an in-depth case study with an empirical dataset and provide JAGS code for carrying out the analysis within the Bayesian framework. The analysis revealed that a subset of the items exhibit partial invariance, and evidence of full invariance was found at the structural level.
"Estimating Bayesian Diagnostic Models with Attribute Hierarchies with the Hamiltonian-Gibbs Hybrid Sampler." Multivariate Behavioral Research, 58(1), pp. 141–142 Article informationConflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.Funding: This work was not supported.Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.Acknowledgment: I would like to express my appreciation to my SMEP sponsor, Jonathan Templin.
Background Throughout the psychotherapeutic and coaching literature, the client-therapist or coach-coachee working alliance has been highlighted as key force driving positive outcome. The Working Alliance Inventory Short form (WAI-S) for coaching charts the quality of working alliance throughout coaching sessions and is broadly applied in coaching research. Due to a shortfall in research on psychometric properties of the WAI-S, the purpose of this study was to examine (a) if the theorized three-factor structure of the 12-item WAI-S forms a solid representation of the dimensions of working alliance in coaching, and (b) longitudinal measurement invariance (LMI) of the WAI-S. Method Data were collected in a two-wave study design comprising a main study sample of N = 690 Dutch coachees that completed the questionnaire at the first measurement, of which N = 490 also completed the questionnaire at the second measurement. Post hoc sensitivity analysis was performed based on the original sample, lacking additional information on covariates, and included both completers and dropouts, comprising N = 1986 respondents at T1, and N = 1020 respondents at T2. Results Confirmatory factor analyses evidenced best fit of the three-factor model in comparison to one-, and two-factor models at both time points. Despite the fact that multigroup confirmatory factor analysis detected non-invariant intercepts, our findings overall supported measurement invariance across coaching sessions. Conclusions As decisions in both clinical and scientific practices generally rely on outcome assessment of interpersonal change in scores on the same measure over time, we believe our findings to be of contributing value to the consolidation of interpretation and accuracy of scorings on the WAI-S in coaching.
Diagnostic classification models (DCMs) are psychometric models for evaluating a student's mastery of the essential skills in a content domain based upon their responses to a set of test items. Currently, diagnostic model and/or Q-matrix misspecification is a known problem with limited avenues for remediation. To address this problem, this paper defines a one-sided score statistic that is a computationally efficient method for detecting under-specification at the item level of both the Q-matrix and the model parameters of the particular DCM chosen in an analysis. This method is analogous to the modification indices widely used in structural equation modeling. The results of a simulation study show the Type I error rate of modification indices for DCMs are acceptably close to the nominal significance level when the appropriate mixture χ2 reference distribution is used. The simulation results indicate that modification indices are very powerful in the detection of an under-specified Q-matrix and have ample power to detect the omission of model parameters in large samples or when the items are highly discriminating. An application of modification indices for DCMs to an analysis of response data from a large-scale administration of a diagnostic test demonstrates how they can be useful in diagnostic model refinement.
Background: Leading eating disorder (ED) theories were informed primarily by samples of White females. Therefore, ED theories lack consideration of sociocultural factors that may impact ED symptom development among Black women. The current study proposed the first culturally informed theory for disordered eating among Black women, positing that ethnic discrimination, strong black woman (SBW) ideology (cultural and societal expectations of strength), and culturally informed appearance satisfaction may significantly impact stress. Stress may be associated with coping-motivated eating behaviors, which may lead to maladaptive weight control behaviors. Methods: Black women (N = 208) completed surveys assessing socio-cultural factors, stress, commensal and binge eating, and maladaptive weight control behaviors. Path analysis was used to test the proposed theory. Results: The final model had a good fit for the data. Findings overall supported the hypothesized model. Specifically, higher ethnic discrimination (beta = 0.044, p = .003), greater endorsement of SBW ideology (beta = 0.074, p =< .001), and lower culturally informed appearance satisfaction (beta =-0.032, p = .025) were associated with greater stress. Stress was positively associated with binge eating (beta = 0.457, p = .046), and binge eating was significantly associated with excessive exercise (=0.152, p = .008) and purging (beta = 0.273, p = <.001). In contrast, commensal eating was not associated with stress or weight control behaviors (p values = .697 to .749). Conclusions: The current study found that stress, as influenced by sociocultural factors, may play a role in binge eating, and subsequently, weight control behaviors among Black women. This theory is a starting point for future research on the specialized conceptualization of eating and maladaptive weight control behaviors among Black women.
In the current paper, we propose a latent interdependence approach to modeling psychometric data in social networks. The idea of latent interdependence is adopted from social relations models (SRMs), which formulate a mutual-rating process by both dyad members' characteristics. Under the framework of the latent interdependence approach, we introduce two psychometric models: The first model includes the main effects of both rating-sender and rating-receiver, and the second model includes a latent distance effect to assess the influence from the dissimilarity between the latent characteristics of both sides. The latent distance effect is quantified by the Euclidean distance between both sides' trait scores. Both models use Bayesian estimation via Markov chain Monte Carlo. How accurately model parameters were estimated was evaluated in a simulation study. Parameter recovery results showed that all parameters were accurately recovered under most of the conditions investigated. As expected, the accuracy of model estimation was significantly improved as network size grew. Also, through analyzing empirical data, we showed how to use the estimates of model parameters to predict the latent weight of connections among group members and rebuild either a univariate or multivariate network at a latent trait level. Finally, we discuss issues regarding model comparison and offer suggestions for future studies.