Rasch/Guttman Scenario (RGS) scales produce scores that map onto interpretable descriptions of individuals at different levels of hierarchically progressive constructs. The unique scenario item format provides actionable and rich content-relevant feedback about (a) respondent status, (b) intervention design, and (c) longitudinal change on a construct. This article presents a seven-step methodological framework for the development of RGS scales. We also reflect on plausible challenges that may arise in the applications of RGS scale development and propose future research directions for the methodology.
Purpose: The purpose of this study was to explore the emotional work of diabetes during emerging adulthood and to explicate the validity of a newly developed measure of diabetes distress (DD) for use with emerging adults living with type 1 diabetes mellitus (T1DM), the Problem Areas in Diabetes—Emerging Adult version (PAID-EA). Methods: Young people ages 18 to 30 with T1DM were recruited online to complete a cross-sectional survey including measures of DD, depressive symptomology, and the PAID-EA. To evaluate content validity, 2 open-ended questions asked what was the most significant emotion or worry discussed in the survey items and what feelings were missed in those items. Responses were analyzed using directed qualitative content analysis. Results: A total of 254 (87%) participants responded to at least 1 of the 2 open-ended questions. Three themes and 1 subtheme were identified: (1) fear of the future with the subtheme of worry about the cost of diabetes, (2) acute worries about living with diabetes, and (3) challenges with finding support. More PAID-EA items corresponded with these themes than items on the original Problem Areas in Diabetes or Center for Epidemiologic Studies Depression Scale, supporting the validity of the PAID-EA and clarifying the developmental-stage-specific aspects of DD. Conclusions: Emerging adulthood is a period in which the future should hold infinite possibility, but young people with T1DM describe a staggering fear of the future with markedly limited possibilities, supporting the need to measure the developmental-stage-specific experience of DD as captured on the PAID-EA.
Mental toughness (MT) predicts outcomes across several high-stress contexts such as athletics, the military, and the workplace. Despite this, researchers have struggled to reach consensus regarding how best to conceptualize and measure MT. MT assessments have focused on measuring general MT rather than domain-specific MT. The current study proposed a measurement model of MT grounded in social-cognitive theory and introduced an assessment of MT within a situational judgment context relevant to the workplace. Participants completed the new MT measure as well as assessments to establish construct validity. Both exploratory and confirmatory factor analyses suggested a three-factor solution fit the data best, consisting of task persistence, emotional control, and utilization of feedback. Cross-structure analyses indicated that the new assessment avoided common-method bias in responding, evidenced by weak correlations with measures of other constructs. The results provided initial evidence to continue research on using a situational judgment test to measure MT.
Accurate parameter estimation in the Rasch model involves the assumption of conditional independence, also termed local independence. Conditional on ability, the responses to items A and B should be independent. Two types of conditional dependence are detailed in this pedagogical piece: trait dependency and response dependency. The bias in difficulty and reliability and the estimates of fit and correlated residuals resulting from these dependencies are compared and contrasted to results from using models that account for the dependency. Contrasts with results from a 2-parameter item response theory model are also briefly noted.
Streiner, Norman and Cairney (2015) "Health Measurement Scales: A practical guide to their development and use", now in its fifth edition, is one of the foundational texts of the health outcomes movement. It states that "the differences between scales constructed with IRT and CTT are trivial." (Streiner, Norman and Cairney, 2015, p. 299) This statement is representative of the view which emphasizes the equivalence of True-Score Theory (TST) (also known as Classical Test Theory [CTT]) and the Rasch Measurement Model [RMM]). This view is widely held and has been one factor in limiting the application of RMM in the development of health outcome measures. However, this equivalence view relies heavily on a paper by Fan (1998) which examined the item statistics derived from TST, IRT (Item Response Theory) and the RMM for a large educational dataset. While subject to a number of theoretical and practical criticisms from a RMM perspective this paper has not been replicated with a large sample. This paper by replicating and extending the paper by Fan (1998) challenges the finding that item difficulty indexes derived from high and low ability samples using TST techniques are invariant. They are not. On the other hand, item locations derived from the RMM have a high degree of invariance. This secondary data analysis, by working through the methods used by Fan (1998) also demonstrates that a reliance on the magnitude of correlational coefficients cannot be used to determine the invariance of item difficulty indexes. An investigation into the linearity of the correlations using scatter plots is also required. Finally, an item analysis derived from the item difficulty indexes which displays a picture of the test as a whole shows that, for this large sample, the differences between scales constructed with TST and the RMM are not trivial.
