Cheating and other threats to score integrity have long challenged language testing, and concerns have intensified with remote proctoring and generative artificial intelligence. Yet prevailing validity frameworks treat test security primarily as an operational matter rather than as a component of validity. We argue that high-stakes testing programs should integrate security explicitly into validity arguments. After outlining how security is treated in prominent work on validity, we propose a distinct security-related Attribution inference within argument-based validation. The Attribution inference claims that observed performance can be credibly attributed to the registered test taker’s unaided, individual effort (unless external materials or collaboration are permitted), a prerequisite for all other inferences. We articulate three warrants supporting this inference, addressing impersonation, unauthorized external assistance, and item pre-knowledge, and specify associated assumptions and potential sources of backing. We further discuss how security vulnerabilities and security measures impact the explanation, utilization, and impact inferences through construct-irrelevant variance, stakeholder trust, and fairness–privacy trade-offs. We conclude that test security evidence should be demonstrated across administration paradigms and reported routinely as part of ongoing validation, with evidentiary expectations depending on test stakes.
Generative AI offers new opportunities for interactive oral communication assessment for specific purposes by building on past research investigating test takers’ oral interaction with spoken dialog systems (SDSs). Performing as automated conversational agents, SDSs can address logistical challenges in classroom-based oral assessments where interactive tasks are difficult to implement. This study investigated the validity of interpretations of a prototype SDS-mediated Tourism English Speaking Test (SDS-TEST) developed using evidence-centered design. Guided by the interpretation/use argument framework, this study focused on supporting the generalization and explanation inferences. Thirty Turkish English as a Foreign Language (EFL) students in a Tourism and Hotel Management program completed three SDS tasks, each rated by four trained raters. The generalization inference was supported using a univariate generalizability study (G-study) with a fully crossed, two-facet design on 360 composite scores, while a decision study (D-study) informed optimal task and rater configurations for future designs. The explanation inference was supported through correlational analyses with related assessments and qualitative insights from participants’ strategy use gathered via stimulated recalls. Findings support the inferences in the validity argument for the SDS-TEST, highlighting the potential of SDSs in English for Specific Purposes oral assessment and offering a model for exploring validity in the next generation of oral communication assessment.
Generative artificial intelligence (GenAI) is advancing at a remarkable speed, and its promise in transforming current practice in language assessment has been articulated by applied linguistics researchers. An emerging application of GenAI is to integrate the technology into Spoken Dialogue Systems (SDSs) to simulate human interlocutors for the purpose of speaking practice or assessment. Despite rapid technological advances, the issue of whether an AI agent can replace a human examiner in one-on-one, high-stakes interactive speaking tests remains contentious. Building on a lively academic debate on this topic at the 2025 Language Testing Research Colloquium (LTRC) in Bangkok, this Viewpoint presents arguments both for and against this proposition in terms of construct theory, practicality, washback, fairness, ethical uses of AI, and so forth.
