● Identification of Innovation: While scholars in rhetoric and composition are showing increasing interest in using large corpora of student writing to analyze student essays and instructor feedback on them, they have largely avoided developing natural language processing technologies designed to provide rich feedback to students on their writing. At the same time, those involved with the development of automated essay evaluation (AEE) systems—who are, almost always, not compositionists—are increasingly developing systems that provide rich feedback on a variety of essay features, rather than simple scores. The current study aims to show an example of how a web-based natural language processing tool can provide students with rich feedback on their papers under a process pedagogy framework.
• Background: This exploratory writing analytics study uses argumentative writing samples from two performance contexts—standardized writing assessments and university English course writing assignments—to compare (1) linguistic features in argumentative writing and (2) relationships between linguistic characteristics and academic performance outcomes. Writing data from this study come from 180 students enrolled at five four-year universities in the United States.
• Background: Researchers interested in quantitative measures of student “success” in writing cannot control completely for contextual factors which are local and site-based (i.e., in context of a specific instructor’s writing classroom at a specific institution). (In)ability to control for curriculum in studies of student writing achievement complicates interpretation of features measured in student writing. This article demonstrates how identifying and analyzing features of writing curriculum can provide dimensions of local context not captured in analysis of student-generated texts alone. Using a dataset of 48 curricular texts collected from 21 instructors teaching in five disciplines across six four-year public universities in the United States, this article: 1) presents a set of curriculum scoring rubrics developed through qualitative analysis, 2) describes a protocol for training raters to use the rubrics to score curricular texts to achieve rater agreement and generate quantitative data, and 3) explores how this framework
This paper provides an overview of how analyses of linguistic features in writing samples provide a greater understanding of predictions of both text quality and writer development and links between language features within texts. Specifically, this paper provides an overview of how language features found in text can predict human judgements of writing proficiency and changes in writing levels in both cross-sectional and longitudinal studies. The goal is to provide a better understanding of how language features in text produced by writers may influence writing quality and growth. The overview will focus on three main linguistic construct (lexical sophistication, syntactic complexity, and text cohesion) and their interactions with quality and growth in general. The paper will also problematize previous research in terms of context, individual differences, and reproducibility
• Background: In the United States, students continue to struggle in the core content areas of language arts, social studies, math, and science. To improve student content learning, writing to learn (WTL) across the content emerged as a potential mechanism. However, few studies examine the extent to which secondary content teachers are ready and willing to implement WTL to improve student learning in those content areas. This study uses structural equation modeling (SEM) to examine how much teaching efficacy, writing apprehension, years teaching, grade level, and content area contribute to teachers’ efficacy of using WTL and their perception of the relevance of WTL to their content areas.
• Background: In this article, we articulate an updated construct describing domains of expertise in writing, one that meets the contemporary needs of those who research, teach, and assess writing—particularly in a digital age. This article appears in a collection published as a special issue of The Journal of Writing Analytics that explores both the challenges and the opportunities involved in integrating digitally delivered formative assessments into classroom instruction, illustrated by the example of Workplace English Communication (WEC). Each article in this special issue addresses different aspects of the challenges involved in developing assessments of complex tasks. The three framework articles that lead this special issue all highlight the importance of robust construct models as a foundation for assessment design. In this article, we present an integrated sociocognitive-oriented construct model for expertise in writing that informs the assessment design work discussed in this special issue.
