This simulation study investigated the recovery of item and person parameters of the one-parameter logistic model for short tests administered to small samples. A potential problem with such small scale testing is the mismatch between item and person location parameter distributions. In our study, we manipulated the match of these distributions as well as test length, sample size, and item discrimination. Results showed the degree of mismatch likely to occur in practice has a relatively modest effect on parameter recovery. As expected, accuracy in parameter estimation decreased as sample size and test length decreased. Nevertheless, researchers investigating small scale tests are likely to view parameter recovery as acceptable if a study has at least 100 subjects and 8 items.

Population and sample simulation approaches were used to compare the performance of parallel analysis using principal component analysis (PA-PCA) and parallel analysis using principal axis factoring (PA-PAF) to identify the number of underlying factors. Additionally, the accuracies of the mean eigenvalue and the 95th percentile eigenvalue criteria were examined. The 95th percentile criterion was preferable for assessing the first eigenvalue using either extraction method. In assessing subsequent eigenvalues, PA-PCA tended to perform as well as or better than PA-PAF for models with one factor or multiple minimally correlated factors; the relative performance of the mean eigenvalue and the 95th percentile eigenvalue criteria depended on the number of variables per factor. PA-PAF using the mean eigenvalue criterion generally performed best if factors were more than minimally correlated or if one or more strong general factors as well as group factors were present.