Stealth assessment: a theoretically grounded and psychometrically sound method to assess, support, and investigate learning in technology-rich environments

Research fields related to learning (e.g., educational technology and learning sciences) have historically focused on what questions using traditional methods (e.g., comparing different learning tools and methods). New methodologies that are grounded in learning, engagement, and motivational theories are needed to additionally address the how questions. Methodologies that use learners’ process data shed more light on how learners learn and if the learning tools are effective, compared to methodologies that use just outcome data. In this paper, we discuss stealth assessment—an evidence-based methodology that can be used in technology-rich environments (e.g., games) to assess and support hard-to-measure constructs (e.g., creativity) as well as knowledge acquisition (e.g., physics). We also discuss evidence-centered design (ECD), and present specific steps to design, embed in a digital learning environment, and evaluate a stealth assessment. Additionally, we provide two examples of stealth assessment studies in the context of an educational game called Physics Playground: Study 1 illustrates a stealth assessment of creativity and Study 2 describes a stealth assessment of physics understanding and how we used it to make the game adaptive. The purpose of this paper is to provide sufficient detail about stealth assessment to help researchers in the field of educational technology and related fields to adopt this method to assess, foster, and investigate learning processes in various technology-rich environments.