The use of standardized test formats in the assessment of historical competencies has recently come under severe criticism, especially in the United States, where standardized tests are particularly common. History researchers have argued that open-ended items are more appropriate for assessment. However, providing largescale evaluations of open-ended answers is time consuming and poses challenges regarding the objectivity, validity, and replicability of ratings. To address this issue, we investigated the extent to which computer-based evaluation methods are suitable for evaluating student answers by combining qualitative methods from history education research with quantitative, computer-based linguistic analyses. In two studies, we analyzed data from an intervention study in which 962 students (ninth graders) completed seven open-ended tasks. In Study 1, we investigated the extent to which task complexity could be predicted from the linguistic complexity of the students' answers. In Study 2, we conducted an automatic content assessment by aligning student answers with predefined target answers and compared automatic scores with human ratings that were based on an elaborate evaluation scheme we developed. In our first study, we identified several linguistic features of students’ answers that successfully predicted task complexity. In our second study, the correlations between manual and computational task scores were encouraging. However, our human interrater agreement left room for improvement and demonstrated the challenges of reliably applying explicit evaluation criteria to open tasks. Overall, our findings illustrate that the combination of qualitative methods from history education research and quantitative computational-linguistic analyses may support the large-scale evaluation of open tasks.
Background: This exploratory study engaged teams of elementary and middle school students in the collaborative design of digital games. Game design is theoretically examined in this study as a form of knowledge-creating learning that is characterized by collaborative efforts to advance a shared object of activity, i.e., the game being designed. Using mixed methods, we examined how students experienced the game design project and how the project fostered connected learning, that is, integration of students' personal interests and supportive peer relations with their schoolwork, and how their self-assessed digital competences developed. Methods: The digital competences of 98 comprehensive school students across Finland were traced using pre-and post-questionnaires. The post-questionnaire also included validated measures on connected learning. Quantitative methods were used to analyze structured measures, and qualitative methods were used to analyze open-ended measures. Findings: Students experienced game design as an inspiring, challenging activity. Game design engaged student teams in sustained, collaborative efforts to create shared digital artifacts. Their efforts involved a great deal of mutual support and knowledge sharing. Participation also improved students' self-reported technical and artistic digital competences. The game design project fostered informal, interest-driven, sociodigital participation; inspired learning engagement; and improved schoolwork practices. Contribution: The game design project appeared to be a pedagogically meaningful way of engaging students in knowledge-creating learning and of connecting students' formal and informal learning. The project sparked students' motivation to learn, fostered digital competences, and enriched the learning environment.
Inquiry-based Learning is increasingly suggested as an efficient approach for fostering learners' curiosity and motivation. It helps learners to develop their ability to work in complex and unpredictable environments making them more critical thinkers and agentic learners. Although mobile technology is a suitable support for this learning process, there is a lack of practical strategies for educational practitioners to enact the right balance between enabling agency and supporting the students through the mobile technology. Thus, we conducted a literature review that analyzed 62 studies on mobile inquiry-based learning. The analysis focused on the level of agency supported by mobile technology. This review study provided two main results. The first result is a two-layer classification –with five types and twelve subtypes– of the most common mobile activities used in inquiry-based learning. The types and subtypes are: 1) Direct instruction formed by 1a) location guidance, 1b) procedural guidance and 1c) metacognitive guidance, 2) Access to content formed by 2a) fixed and 2b) dynamic content, 3) Data collection that consists of 3a) cooperative and 3b) collaborative data collection, 4) Peer-to-peer communication formed by 4a) asynchronous and 4b) synchronous social communications and 5) Contextual support that includes 5a) augmented experience, 5b) immersive experience and 5c) adaptive feedback. The second result consists of an analytical framework –based on six dimensions– to assess the level of agency supported by the different types of mobile activities. The learners' agency dimensions are: 1) Goals, 2) Content, 3) Actions, 4) Strategies, 5) Reflection and 6) Monitoring. Finally, the review presents insights on how this analytical framework can be used by educational practitioners to identify mobile activities that effectively balance learners’ agency with mobile technology.
The increased use of online discussions in learning environments both formal and informal, positions the construct of interactivity as central to learning. Interactivity in learning communities' online discourse is viewed in this study as a socio-constructivist process. It is the network of interactions among content items and participants which drives a collective knowledge construction process. Conceptualizing interactivity in the literature is still unclear and not enough is known about its role in knowledge construction and about its relationship to learning outcomes. In addition, assessing learning outcomes using analytics has not matured fully and is still subject to intense development. This study thus sets out to investigate the role of interactivity as a process of knowledge construction within online discussions, and in particular, its association with learning outcomes, as measured by formal assessment tasks. We present significant positive correlations between various interactivity measures, taken from various learning communities, and a set of well-known learning assessments. We suggest that patterns of interactivity among learners can be measured, and teach us, not just about group dynamics and collaboration, but also about the actual individual learning process.
