Measurement in psychology is often associated with psychometrics. Despite the theoretical insights afforded by psychometric methods, they represent only a restricted range of possible measurement approaches in the social sciences. Representational measurement theory (RMT) generalizes the logic of measurement developed in the physical sciences to domains where direct concatenation is not available, making it particularly relevant for psychological research. However, RMT remains unfamiliar to many researchers, largely because of its abstract mathematical formulation and the gap between its formal results and applied practice. This paper provides an accessible introduction to RMT that emphasizes intuitive understanding while offering a structured pathway for quantitatively inclined readers to engage with its technical foundations. We first present a general description of RMT, followed by its four central problems: representation, uniqueness, scaling and meaningfulness. Finally, we discuss recent developments, including probabilistic extensions of RMT, their relationship to statistical modelling and their implications for the coordination between theoretical constructs and observed behaviour.
Item response theory (IRT) has been a prominent modelling framework in educational and psychological measurement. Traditional IRT models are bipolar, commonly assuming symmetric measurement link functions and symmetric trait distributions over a latent continuum unbounded at both ends. However, many measured constructs, such as print exposure, vocabulary knowledge, or addictive behaviours, have been argued to be unipolar. While prior unipolar applications have emphasized the potential meaningfulness of measurement metrics having fixed lower bounds, a less-explored feature relates to how the unipolar framework has the potential to represent increased diversification of the construct as the construct level increases. This paper illustrates this feature by considering a form of multidimensional unipolar IRT modelling developed using an Accelerated Failure Time (AFT) framework. Using empirical data from the Author Recognition Test-Genres (ART-G), we show how multidimensional unipolar models can help represent such construct diversification. In the context of reading assessment, we show how such a representation has the potential to explain the diversification of vocabulary mastery as the vocabulary proficiency develops.
Generalizability theory (GT) provides a variety of intraclass correlation coefficients (ICCs) to quantify reliability across different conditions of measurement-for example, scale and interrater reliability quantify generalization across items and raters, respectively. In a multilevel modelling (MLM) context, an ICC is often used to quantify the proportion of a subject-level variable's variance that is attributable to cluster-level differences (e.g., between schools or neighbourhoods). When data from cluster members are their ratings of a group-level characteristic, the ICC is an inter-rater reliability coefficient, interpreted as the degree to which observations of a cluster generalize across members of that cluster. Structural equation modelling (SEM) is becoming a popular framework for modelling GT, and the 'omega' indices of composite reliability-which are functions of SEM parameters-are special cases of GT (i.e., how measurements of a construct generalize across indicators). The merger of MLM and SEM (i.e., ML-SEM) provides a unique opportunity to explore how well measurements of cluster-level constructs generalize across multiple measurement conditions (indicators, cluster members, or both). A taxonomy of ICCs is explicated to appropriately quantify reliability across aspects of a procedure for measuring cluster-level constructs, adding value to the limited ICCs currently considered in ML-SEM applications.
Amortized variational inference (AVI) has become a popular, computationally efficient alternative to marginal maximum likelihood estimation (MML) for estimating high-dimensional item response theory models on large datasets. However, currently the AVI framework lacks methods to assess model fit and determine parameter uncertainty. Therefore, it would be desirable to have the well established, classical goodness-of-fit based modelling assessment tools from item response theory available to assess models fit with AVI. However, AVI optimizes a lower-bound to the marginal likelihood, and the asymptotic properties that are required to use these tools are not guaranteed. In this paper, we use AVI estimates in a likelihood-based Fisher scoring procedure, ensuring asymptotic guarantees associated with MML. Large-sample theory for one-step estimators shows that when the initial estimator is sufficiently accurate (within an Op(N)$$ {O}_p\left(\sqrt{N} ight) $$ neighbourhood of the true parameter), a single Fisher scoring update is sufficient to obtain a consistent, and asymptotically efficient estimator. We investigate empirically whether one scoring step suffices to restore the validity of classical goodness-of-fit measures and standard error estimation. Using simulation studies and an application to large-scale Narcissism Personality Inventory data, we show that the proposed AVI-initialized one-step Fisher scoring approach yields fit statistics and uncertainty estimates that are on par with MML while retaining the computational advantage of AVI.
