Since its founding in 2011, Kidaptive has built customized models that provide adaptivity and/or personalization in online learning environments. We have supported adaptive game-based learning through rule-based and dynamic Bayesian psychometric models, and we have developed behavioral models for online learning and online test preparation environments based on learners’ time management, answer behavior, and test scores. Our models are deployed on a scalable distributed-computing platform that has supported millions of learners, but the human expertise required to build custom models for every learning environment is not scalable. To address this limitation, we have recently been working toward an abstracted version of our psychometric and behavioral models, to be provided as an “out-of-the-box” product offering. This paper describes insights and challenges encountered in this process.

This chapter closes by acknowledging the need for empirical research that can further inform personalized and adaptive learning game design, preparing the reader for a deeper discussion of measurement and evaluation in the closing chapter of this book. In this chapter, the authors focus on Kidaptive's proprietary Adaptive Learning Platform, which uses Bayesian Item Response Theory to equip game designers with the observations and interventions needed to support learner variation. To better understand how adaptive leveling works, it is best to play "Fish Force" to the 20th level or beyond. Although formative testing indicates that "Fish Force" can engage children for longer, focused play, when positioned alongside other options, many learners might not stick with it long enough to actually get to adaptive levels. Formative testing demonstrated that players sought feedback on their progress, an important consideration for future personalized and adaptive learning games.

Video games are pervasive in today's culture, and the time kids spend playing them may seem, from a teacher's perspective, as time that is lost to education. Sometimes, sadly, this is true. But many good video games, although not explicitly educational in focus, provide powerful experiences that are rich fodder for subsequent instruction. Looking at game play this way-as a source of rich prior experiences that can be brought to bear to illuminate curricular concepts-frees teachers to explore the potential benefits of having a classroom full of gamers.

Well-designed digital games can deliver powerful experiences that are difficult to provide through traditional instruction, while traditional instruction can deliver formal explanations that are not a natural fit for gameplay. Combined, they can accomplish more than either can alone. An experiment tested this claim using the topic of statistics, where people’s everyday experiences often conflict with normative statistical theories and a videogame might provide an alternate set of experiences for students to draw upon. The research used a game called Stats Invaders!, a variant of the classic videogame Space Invaders. In Stats Invaders!, the locations of descending alien invaders follow probability distributions, and players need to infer the shape of the distributions to play well. The experiment tested whether the game developed participants’ intuitions about the structure of random events and thereby prepared them for future learning from a subsequent written passage on probability distributions. Community-college students who played the game and then read the passage learned more than participants who only read the passage.

An argument that choice-based, process-oriented educational assessments are more effective than static assessments of fact retrieval.If a fundamental goal of education is to prepare students to act independently in the world—in other words, to make good choices—an ideal educational assessment would measure how well we are preparing students to do so. Current assessments, however, focus almost exclusively on how much knowledge students have accrued and can retrieve. In Measuring What Matters Most, Daniel Schwartz and Dylan Arena argue that choice should be the interpretive framework within which learning assessments are organized. Digital technologies, they suggest, make this possible; interactive assessments can evaluate students in a context of choosing whether, what, how, and when to learn.Schwartz and Arena view choice not as an instructional ingredient to improve learning but as the outcome of learning. Because assessments shape public perception about what is useful and valued in education, choice-based assessments would provide a powerful lever in this reorientation in how people think about learning.Schwartz and Arena consider both theoretical and practical matters. They provide an anchoring example of a computerized, choice-based assessment, argue that knowledge-based assessments are a mismatch for our educational aims, offer concrete examples of choice-based assessments that reveal what knowledge-based assessments cannot, and analyze the practice of designing assessments. Because high variability leads to innovation, they suggest democratizing assessment design to generate as many instances as possible. Finally, they consider the most difficult aspect of assessment: fairness. Choice-based assessments, they argue, shed helpful light on fairness considerations.

If a fundamental goal of education is to prepare students to act independently in the world -- in other words, to make good choices -- an ideal educational assessment would measure how well we are preparing students to do so. Current assessments, however, focus almost exclusively on how much knowledge students have accrued and can retrieve. In Measuring What Matters Most, Daniel Schwartz and Dylan Arena argue that choice should be the interpretive framework within which learning assessments are organized. Digital technologies, they suggest, make this possible; interactive assessments can evaluate students in a context of choosing whether, what, how, and when to learn. Schwartz and Arena view choice not as an instructional ingredient to improve learning but as the outcome of learning. Because assessments shape public perception about what is useful and valued in education, choice-based assessments would provide a powerful lever in this reorientation in how people think about learning. Schwartz and Arena consider both theoretical and practical matters. They provide an anchoring example of a computerized, choice-based assessment, argue that knowledge-based assessments are a mismatch for our educational aims, offer concrete examples of choice-based assessments that reveal what knowledge-based assessments cannot, and analyze the practice of designing assessments. Because high variability leads to innovation, they suggest democratizing assessment design to generate as many instances as possible. Finally, they consider the most difficult aspect of assessment: fairness. Choice-based assessments, they argue, shed helpful light on fairness considerations.

We propose that adoption of game-based-learning principles can be increased by providing standardized-test evidence of learning from gameplay. This paper describes a game called Tug-of-War as a candidate for such evidence. Tug-of-War is designed to help fourth-grade students build fluency with fractions. Development of the game followed an iterative design process of user testing and rules refinement, culminating in an experimental trial in which a single fourth-grade class was divided into two cohorts. Each cohort played Tug-of-War for six or seven weekly 75-minute sessions while the other cohort participated in unrelated research. Results indicate that both cohorts achieved significant learning gains by playing Tug-of-War in addition to the traditional curriculum. Playing Tug-of-War was also shown to significantly improve scores on the fractions subsection of a statewide standardized test.

In this worked example, I propose a framework for characterizing the learning that occurs in particular uses of a game for educational purposes. The framework is based upon the ancient Greek rhetorical structure that evolved into the "five W's" of modern journalism (who, what, when, where, why--the Greeks also had "how" and "with what"). This framework is just one (worked) example of how we might achieve the larger aim of this proposal, which is to encourage game designers and researchers to be explicit about the theories of learning, goals, and contexts that undergird their designs or analyses, which might help the field of game-based learning research to develop a common language and facilitate exploration of the many regions of what I consider a high-dimensional space. In this example I describe the seven circumstances of game-based learning and offer examples of how we might locate particular games within this space.