Investigating the Cross-National Comparability of Testing Using Response Times

The cross-national comparability of testing results is one of the main concerns associated with international assessment. One way to approach this concern is the theoretical framework of cross-national comparability. One such framework, proposed by Van de Vijver and Tanzer [1997], describes the bias in item meaning across countries and the bias in terms of test administration procedures and respondent behavior across countries. To study how items function in terms of respondent behavior and in terms of their psychological meaning, we introduce the concept of time-related differential item functioning (DIF). This concept is similar to traditional DIF but describes the incomparable time parameters of items across countries rather than item difficulty parameters. We discuss time-related DIF, referencing dual processing cognitive theory and illustrate how it limits the interpretability of a certain type of time-related DIF. We also demonstrate the analysis with real data, and discuss the difference introduced by response time information in parameter interpretation and modelling results.