We explored relations between reading comprehension performance and self-reported components of metacognition in middle-school children. Students' self-reported metacognitive strategies in planning and evaluation accounted for significant variance in reading comprehension performance on questions involving inferences. In Study 2, middle school students read a science text then made predictions about how they would perform on a comprehension test. Students' metacomprehension accuracy was related to their performance at different levels of understanding. Students' text-based question performance accounted for significant variance in metacomprehension accuracy for text-based questions, and inference-based question performance accounted for significant variance in metacomprehension accuracy for inference-based questions. Results from the two studies suggest that metacognitive and metacomprehension knowledge is aligned with the level of information given in text, and is related to deeper understanding of texts, particularly for inferential information. We discuss the implications of these findings and how future research on absolute metacomprehension accuracy should consider different levels of understanding.

This chapter describes development efforts that build upon the Interactive Strategy Trainer for Active Reading and Thinking-2 (iSTART-2), an intelligent tutoring system that provides self-explanation strategy instruction to improve reading comprehension. The chapter reflects on considerations of the unique needs of adult literacy learners, and outlines the specific guidelines followed to adapt the system to these learners. Several modifications have been made to adapt iSTART to adult learners, including the following: 1) two additional strategy instructional modules for summarization and deep question asking, 2) a text library with life-relevant texts for adult learners, and 3) an interactive narrative which allows instantiated practice of reading strategies using life-relevant artifacts. The authors also describe results from two attitudinal studies examining learners' perceptions of the interactive narrative.

The purpose of this research endeavor was to develop and validate a new measurement tool predicated on previous research to assess learners’ metacomprehension during reading. In two separate studies (N = 923) with Chilean undergraduate students, we demonstrate the versatility and utility of our proposed Metacomprehension Inventory (MI). In Study 1, we provide empirical support for the psychometric soundness and construct validity of the MI. In Study 2, we provide evidence of the measurement invariance of the MI between males and females. Results of Study 1 revealed the hypothesized factor structure of the MI is sound, with high factor loadings, excellent model fit, and moderate-to-strong inter-factor correlations. Study 2 results indicated that the MI is interpreted similarly by both males and females, as factor loadings were largely statistically identical across the two groups. We discuss implications of our proposed MI for theory and applied research.

iSTART is an intelligent tutoring system designed to provide self-explanation instruction and practice to improve students' comprehension of complex, challenging text. This study examined the effects of extended game-based practice within the system as well as the effects of two metacognitive supports implemented within this practice. High school students (n = 234) were either assigned to an iSTART treatment condition or a control condition. Within the iSTART condition, students were assigned to a 2 × 2 design in which students provided self-assessments of their performance or were transferred to Coached Practice if their performance did not reach a certain performance threshold. Those receiving iSTART training produced higher self-explanation and inference-based comprehension scores. However, there were no direct effects of either metacognitive support on these learning outcomes.

iSTART is a web-based reading comprehension tutor. A recent translation of iSTART from English to Spanish has made the system available to a new audience. In this paper, we outline several challenges that arose during the development process, specifically focusing on the algorithms that drive the feedback. Several iSTART activities encourage students to use comprehension strategies to generate self-explanations in response to challenging texts. Unsurprisingly, analyzing responses in a new language required many changes, such as implementing Spanish natural language processing tools and rebuilding lists of regular expressions used to flag responses. We also describe our use of an algorithm inspired from genetics to optimize the Fischer Discriminant Function Analysis coefficients used to determine self-explanation scores.

In this chapter, we describe several intelligent tutoring systems (ITSs) designed to support student literacy through reading comprehension and writing instruction and practice. Although adaptive instruction can be a powerful tool in the literacy domain, developing these technologies poses significant challenges. For example, evaluating the quality of a student's writing can be challenging because of the numerous ways to succeed (or fail) when generating a written work. Throughout our discussion, we focus on the methodologies that ITSs have employed to face these challenges. Natural language processing techniques, for example, can be leveraged to assess students' level of comprehension or writing proficiency and subsequently drive the feedback that students receive. Additional challenges arise in the implementation of these systems in classrooms; we discuss how the features and flexibility offered by ITSs can augment their usefulness in these real-world settings. We conclude the chapter by forecasting how future generations of ITSs for literacy will improve and fit into the educational landscape.

This study investigates how and whether information about students’ writing can be recovered from basic behavioral data extracted during their sessions in an intelligent tutoring system for writing. We calculate basic and time-sensitive keystroke indices based on log files of keys pressed during students’ writing sessions. A corpus of prompt-based essays was collected from 126 undergraduates along with keystrokes logged during the session. Holistic scores and linguistic properties of these essays were then automatically calculated using natural language processing tools. Results indicated that keystroke indices accounted for 76% of the variance in essay quality and up to 38% of the variance in the linguistic characteristics. Overall, these results suggest that keystroke analyses can help to recover crucial information about writing, which may ultimately help to improve student models in computer-based learning environments.

This paper reports development of an interactive narrative to be added to the existing intelligent tutoring system, the Interactive Strategy Trainer for Active Reading and Thinking-2 (iSTART-2). The design of this module is specifically tailored for adult learners who read below functional literacy levels. Little research focuses on the educational needs of adult literacy learners, but available evidence indicates they require support for both lower-and higher-order reading skills. The interactive narrative provides varied practice opportunities supporting these skills and uses life-relevant texts which hold personal significance for adult readers. We describe the design and development of the interactive narrative, and preliminary results from a small pilot study indicating overall enjoyment of the narrative.

This chapter provides an overview of the Interactive Strategy Tutor for Active Reading and Thinking-2 (iSTART-2). iSTART-2 is a game-based tutoring system designed to improve students' reading comprehension skills. It discusses why reading comprehension is a critical skill and how the iSTART-2 system addresses the development of this skill through the instruction of comprehension strategies. The chapter also provides an overview of the iSTART-2 system and its features. It also discusses the need for this reading comprehension technology in the classroom and provides a general overview of how iSTART-2 addresses those needs and the Common Core State Standards. The chapter describes previous findings from research using the various iterations of the iSTART-2 program. The iSTART-2 program provides students with instruction on comprehension strategies that help them to overcome gaps in their domain knowledge and achieve deep level comprehension of the material within the text. The chapter also describes how iSTART-2 can be accessed by teachers and used within classroom environments.

Revising is an essential writing process yet automated writing evaluation systems tend to give feedback on discrete essay drafts rather than changes across drafts. We explore the feasibility of automated revision detection and its potential to guide feedback. Relationships between revising behaviors and linguistic features of students’ essays are discussed.

Game-based practice within Intelligent Tutoring Systems (ITSs) can be optimized by examining how properties of practice activities influence learning outcomes and motivation. In the current study, we manipulated when game-based practice was available to students. All students (n = 149) first completed lesson videos in iSTART-2, an ITS focusing on reading comprehension strategies. They then practiced with iSTART-2 for two 2-hour sessions. Students' first session was either in a game or nongame practice environment. In the second session, they either switched to the alternate environment or remained in the same environment. Students' comprehension was tested at pretest and posttest, and motivational measures were collected. Overall, students' comprehension increased from pretest to posttest. Effect sizes of the pretest to posttest gain suggested that switching from the game to nongame environment was least effective, while switching from a nongame to game environment or remaining in the game environment was more effective. However, these differences between the practice conditions were not statistically significant, either on comprehension or motivation measures, suggesting that for iSTART-2, the timing of game-based practice availability does not substantially impact students' experience in the system.

This study investigates how cohesion manifests in readers’ thought processes while reading texts when they are instructed to engage in self-explanation, a strategy associated with deeper, more successful comprehension. In Study 1, college students (n = 21) were instructed to either paraphrase or self-explain science texts. Paraphrasing was characterized by greater cohesion in terms of lexical overlap whereas selfexplanation included greater lexical diversity and more connectives to specify relations between ideas. In Study 2, adolescent students (n = 84) were provided with instruction and practice in self-explanation and reading strategies across 8 sessions. Self-explanations increased in lexical diversity but became more causally and semantically cohesive over time. Together, these results suggest that cohesive features expressed in think alouds are indicative of the depth of students’ comprehension processes.

Writing training systems have been developed to provide students with instruction and deliberate practice on their writing. Although generally successful in providing accurate scores, a common criticism of these systems is their lack of personalization and adaptive instruction. In particular, these systems tend to place the strongest emphasis on delivering accurate scores, and therefore, tend to overlook additional indices that may contribute to students' success, such as their affective states during writing practice. This study takes an initial step toward addressing this gap by building a predictive model of students' affect using information that can potentially be collected by computer systems. We used individual difference measures, text indices, and keystroke analyses to predict engagement and boredom in 132 writing sessions. The results suggest that these three categories of indices were successful in modeling students' affective states during writing. Taken together, indices related to students' academic abilities, text properties, and keystroke logs were able classify high and low engagement and boredom in writing sessions with accuracies between 76.5% and 77.3%. These results suggest that information readily available in writing training systems can inform affect detectors and ultimately improve student models within intelligent tutoring systems.

Writing researchers have suggested that students who are perceived as strong writers (i.e., those who generate texts that are rated as high quality) demonstrate flexibility in their writing style. While anecdotally this has been a commonly held belief among researchers, scientists, and educators, there is little empirical research to support this claim. This study further investigates this hypothesis by examining how students vary in their use of two linguistic features (i.e., narrativity and cohesion) across 16 prompt-based essays. Forty-five high school students wrote 16 essays across 8 sessions within an Automated Writing Evaluation (AWE) system. Natural language processing (NLP) techniques and Entropy analyses were used to calculate how rigid or flexible students were in their use of narrative and cohesive linguistic features over time and how this trait related to individual differences in literacy abilities (i.e., vocabulary knowledge and comprehension ability), prior world knowledge, and essay quality. For instance, through the unique combination of NLP and Entropy, we found that patterns of narrative flexibility (or rigidity) was significantly and reliably related to students’ prior reading comprehension ability after 2 sessions (4 essays). Conversely, students’ flexible (or rigid) use of cohesive features was reliably related to their prior reading comprehension ability after 5 sessions (10 essays). These exploratory methodologies are important for researchers and educators, as they indicate that writing flexibility is indeed a trait of strong writers and can be detected rather quickly using the combination of textual features and dynamic analyses.

Work in cognitive and educational psychology examines a variety of phenomena related to the learning and retrieval of information. Indeed, Alice Healy, our honoree, and her colleagues have conducted a large body of groundbreaking research on this topic. In this article we discuss how 3 learning principles (the generation effect, deliberate practice and feedback, and antidotes to disengagement) discussed in Healy, Schneider, and Bourne (2012) have influenced the design of 2 intelligent tutoring systems that attempt to incorporate principles of skill and knowledge acquisition. Specifically, this article describes iSTART-2 and the Writing Pal, which provide students with instruction and practice using comprehension and writing strategies. iSTART-2 provides students with training to use effective comprehension strategies while self-explaining complex text. The Writing Pal provides students with instruction and practice to use basic writing strategies when writing persuasive essays. Underlying these systems are the assumptions that students should be provided with initial instruction that breaks down the tasks into component skills and that deliberate practice should include active generation with meaningful feedback, all while remaining engaging. The implementation of these assumptions is complicated by the ill-defined natures of comprehension and writing and supported by the use of various natural language processing techniques. We argue that there is value in attempting to integrate empirically supported learning principles into educational activities, even when there is imperfect alignment between them. Examples from the design of iSTART-2 and Writing Pal guide this argument.