In the continuous administration framework of computerized adaptive testing (CAT), item preknowledge poses a significant threat to test fairness and security of the item bank. Traditional person-fit statistics (e.g., l z ) often fail in CAT due to their reliance on asymptotic normality assumptions that are easily violated in short tests. While cumulative sum (CUSUM) methods are better suited for dynamic testing environments, they still face limitations, including insufficient detection power in short tests, the lack of optimization for preknowledge-specific patterns in standardized residual statistics, and an inherent mismatch with the non-continuous nature of preknowledge behaviors. This study proposes a new weighted residual statistic that amplifies abnormal signals by weighting positive residuals and introduces the posterior predictive model checking (PPMC) framework. By simulating examinees’ personalized testing processes to generate posterior distributions, PPMC establishes individualized critical values, providing a precise and adaptive diagnostic standard for CAT environments. Both simulation and empirical studies demonstrate that the weighted residual statistic, integrated with PPMC, effectively optimizes Type I error rates and exhibits higher detection power across various testing conditions compared to traditional statistics. This study provides a sensitive, robust, and practical solution for operational security monitoring in CAT.
Two common methods for modeling abilities that change over time are generalized linear mixed models (GLMMs) and the Elo algorithm. GLMM specifications that avoid assumptions about the shape of the growth curve tend to be computationally heavy. The Elo algorithm is both flexible and computationally inexpensive but produces estimates that tend to lag when the true latent state changes rapidly from one response to the next, as can occur when the responses are sparse over time. We propose the Elo-informed growth model, a hybrid method that uses the Elo algorithm to rank respondents and combines this with the assumption that the abilities at each iteration in a sequence of responses follow a distribution that can be estimated. Using both simulated data and real learning data, we show that the proposed method performs better than the standard Elo algorithm under rapid ability growth. It achieves accuracy similar to a flexible GLMM while requiring less training data and a fraction of the fitting time. This combination lowers the threshold for implementing robust ability tracking.
Classification consistency evaluates the degree of consistency of the classifications based on observed scores from repeated testing. The classification consistency index is often used in practice for tests that categorize examinees into two or more categories with respect to a specific standard. In this paper, the asymptotic standard error of the classification consistency index under item response theory (IRT) that utilizes raw scores was derived using the delta method for the three-parameter logistic (3PL) model. The derived formula was applied to real data sets, and its accuracy was examined using simulated data sets under various dichotomous IRT models, sample sizes, and latent densities. In general, the asymptotic standard errors were mostly accurate for the 3PL and 2PL models. The formula also worked well for the 1PL model if multiple cut scores were applied at the same time or a single cut score that was not close to the mean score was used to make binary decisions.
Haberman’s value-added approach is widely used to evaluate the added value of subscores. However, it relies on parameter estimates from sample data and does not account for sampling variability. To address this, Sinharay proposed applying statistical tests to an extended interpretation of the value-added method. Yet, the assumptions underlying these tests may be violated in the context of added value, and a systematic evaluation of their performance has been lacking. This paper examines Sinharay’s approach and introduces two alternatives: applying the tests to the original value-added method and using a non-parametric bootstrap procedure. Simulation results show that the parametric methods fail to adequately control Type I error rates, while the bootstrap procedure offers a more viable alternative.
The growing scale and complexity of statistical and psychometric analyses call for tools that are efficient, flexible, and accessible within the R ecosystem. This article presents R , a high-performance tensor computing framework with automatic differentiation that brings core PyTorch capabilities to R. We offer a hands-on tutorial that demonstrates vectorized computation, automatic differentiation, and Graphics Processing Unit (GPU) acceleration for common tasks, with case studies showing concise code and substantial gains in speed and scalability over base R. Practical guidance for reliable, readable, and reproducible implementation is included. This tutorial emphasizes conceptual clarity and reproducibility, positioning R as a bridge between traditional statistical workflows and modern large-scale computational methods.
