Number 26 Assessing IRT Model-Data Fit for Mixed Format Tests ∗

This study examined various model combinations and calibration procedures for mixed format tests under different item response theory (IRT) models and calibration methods. Using real data sets that consist of both dichotomous and polytomous items, nine possibly applicable IRT model mixtures and two calibration procedures were compared based on traditional and alternative goodnessof-fit statistics. Three dichotomous models and three polytomous models were combined to analyze mixed format test using both simultaneous and separate calibration methods. To assess goodness of fit, The PARSCALE’s G was used. In addition, two fit statistics proposed by Orlando and Thissen (2000) were extended to more general forms to enable the evaluation of fit for mixed format tests. The results of this study indicated that the three parameter logistic model combined with the generalized partial credit model among various IRT model combinations led to the best fit to the given data sets, while the one parameter logistic model had the largest number of misfitting items. In a comparison of three fit statistics, some inconsistencies were found between traditional and new indices for assessing the fit of IRT models to data. This study found that the new indices indicated considerably better model fit than the traditional indices.