Background During the past decade, the increasingly heterogeneous field of learning analytics has been critiqued for an over-emphasis on data-driven approaches at the expense of paying attention to learning designs. Method and objective In response to this critique, we investigated the role of learning design in learning analytics through a systematic literature review. 161 learning analytics (LA) articles were examined to identify indicators that were based on learning design events and their associated metrics. Through this research, we address two objectives. First, to achieve a better alignment between learning design and learning analytics by proposing a reference framework, where we present possible connections between learning analytics and learning design. Second, to present how LA indicators and metrics have been researched and applied in the past. Results and conclusion In our review, we found that a number of learning analytics papers did indeed consider learning design activities for harvesting user data. We also found a consistent increase in the number and quality of indicators and their evolution over the years.

Learning in asynchronous online settings (AOSs) is challenging for university students. However, the construct of learning engagement (LE) represents a possible lever to identify and reduce challenges while learning online, especially, in AOSs. Learning analytics provides a fruitful framework to analyze students' learning processes and LE via trace data. The study, therefore, addresses the questions of whether LE can be modeled with the sub-dimensions of effort, attention, and content interest and by which trace data, derived from behavior within an AOS, these facets of LE are represented in self-reports. Participants were 764 university students attending an AOS. The results of best-subset regression analysis show that a model combining multiple indicators can account for a proportion of the variance in students' LE (highly significant R2 between 0.04 and 0.13). The identified set of indicators is stable over time supporting the transferability to similar learning contexts. The results of this study can contribute to both research on learning processes in AOSs in higher education and the application of learning analytics in university teaching (e.g., modeling automated feedback).

Learning Analytics Dashboards (LAD) have been developed as feedback tools to help students self-regulate their learning (SRL), using the large amounts of data generated by online learning platforms. Despite extensive research on LAD design, there remains a gap in understanding how learners make sense of information visualised on LADs and how they self-reflect using these tools. We address this gap through an experimental study where a LAD delivered personalised SRL feedback based on interactions and progress to a treatment group, and minimal feedback based on the average scores of the class to a control group. Following the feedback, students were asked to state in writing how they would change their study behaviour. Using a coding scheme covering learning strategies, metacognitive strategies and learning materials, three human coders coded 1,251 self-reflection texts submitted by 417 students at three time points. Our results show that learners who received personalised feedback intend to focus on different aspects of their learning in comparison to the learners who received minimal feedback and that the content of the dashboard influences how students formulate their self-reflection texts. Based on our findings, we outline areas where support is needed to improve learners' sense-making of feedback on LADs and self-reflection in the long term.

Reminders are a popular feature in smartphone apps designed to promote desirable behaviors that are best performed regularly. But can they also promote students' regular studying? In the present study with 85 lower secondary school students aged 10-12, we combined a smartphone-based between- and within-person experimental manipulation with logfile data of a vocabulary learning app. Students were scheduled to receive reminders on 16 days during the 36-day intervention period. Findings suggest that reminders can be a double-edged sword. The within-person experimental manipulation allowed a comparison of study probability on days with and without reminders. Students were more likely to study on days they received a reminder compared to days when they did not receive a reminder. However, when compared to a control group that never received reminders, the effect was not due to students studying more frequently on days with reminders. Instead, they studied less frequently on days without reminders than students in the control group. This effect increased over the study period, with students becoming increasingly less likely to study on days without reminders. Taken together, these results suggest a detrimental side effect of reminders: students become overly reliant on them.

Distractions are ubiquitous in today’s technology-saturated environments, an issue that significantly impacts learning contexts employing digital technologies and yields detrimental effects on learning. Digital self-control tools, which aim to assist users in their efforts to reduce digital distractions, are numerous and readily available. Despite several dedicated empirical studies focusing on specific tools, there remains a notable lack of information regarding their daily use and helpfulness. Furthermore, the sheer variety of these tools prompts questions about their universal helpfulness and the potential influence of individual differences. To address these issues, we surveyed a sample of higher-education students, totaling 273 individuals. These students reported on their media use, satisfaction with learning, and experiences with features of digital self-control tools. Our study’s findings indicate a discrepancy in the perception and awareness of these features; those deemed most helpful are among the least known, and conversely, common features are often perceived as unhelpful. Our research also uncovered a negative correlation between habitual media use and the use of less restrictive features. Another identified issue was constraints on the use of these tools for learning, as platforms often serve dual purposes for both education and entertainment. We delve into these practical problems and propose future research directions to further advance the understanding of digital self-control tools.

Learners use digital media during learning for a variety of reasons. Sometimes media use can be considered “on-task,” e.g., to perform research or to collaborate with peers. In other cases, media use is “off-task,” meaning that learners use content unrelated to their current learning task. Given the well-known problems with self-reported data (incomplete memory, distorted perceptions, subjective attributions), exploring on-task and off-task usage of digital media in learning scenarios requires logging activity on digital devices. However, we argue that logging on- and off-task behaviour has challenges that are rarely addressed. First, logging must be active only during learning. Second, logging represents a potential invasion of privacy. Third, logging must incorporate multiple devices simultaneously to take the reality of media multitasking into account. Fourth, logging alone is insufficient to reveal what prompted learners to switch to a different digital activity. To address these issues, we present a contextually activated logging system that allows users to inspect and annotate the observed activities after a learning session. Data from a formative study show that our system works as intended, and furthermore supports our assumptions about the diverse intentions of media use in learning. We discuss the implications for learning analytics.

ABSTRACT Educational applications (apps) offer opportunities for designing learning activities children enjoy and benefit from. We redesigned a typical mobile learning activity to make it more enjoyable and useful for children. Relying on the technology acceptance model, we investigated whether and how implementing this activity in an app can increase children's intention to use. During the 27‐day study, children ( N = 103, 9–14 years) used the app to memorize one‐sentence learning plans each day. Children used three different app‐based learning activities throughout the study. In two standard activities, children reread or reassembled the words of the plan. In the redesigned activity, children represented the meaning of the plan with emojis. Children repeatedly reported on their attitude towards each activity. Subsequently, children reported perceived enjoyment and intention to use the app. Results showed children found the emoji activity most enjoyable, and enjoyment of the emoji activity contributed uniquely towards intention to use. Additionally, children's enjoyment of the app mediated their intention to use the app in the future. Overall, the study suggests that children's enjoyment of an app is crucial in predicting their subsequent intention to use, and it provides a concrete example of how emojis can be used to boost enjoyment. Practitioner notes What is already known about this topic Educational applications provide children with unrestricted access to mobile learning resources. Positive attitudes towards educational applications predict behavioural intention to use these applications, at least in young adults. There is a need for more research examining the relevance of enjoyable learning activities in fostering children's sustained usage of an educational application. What this paper adds Positive attitude towards the use of emojis during learning activities uniquely contributed to children's behavioural intention to use the application. Perceived enjoyment predicted behavioural intention to use the application. Perceived enjoyment mediated the effect of attitude towards using learning activities on the behavioural intention to use the mobile educational application. Implications for practice and/or policy These findings highlight the importance of enjoyment for children's' acceptance of educational applications. Enjoyable learning activities are necessary to ensure sustained usage of educational applications. The paper provides a concrete example of how emojis can be used to boost enjoyment of a typical mobile learning activity.

Planning is an important but difficult self-regulation strategy. Successful implementation of a plan requires that it can be recalled when needed in everyday life. Children in particular are unlikely to internalize plans effectively. Therefore, we developed PROMPT, a planning app for children. We designed three learning activities, including a passive reading activity, an active recall activity, and a generative activity. Children (N = 106, 9-14 years) used PROMPT for 27 days to memorize one plan per day, alternating between these activities. Unexpectedly, neither the active recall activity nor the generative activity was associated with better overall recall than the passive activity. Only children who spent more time on these activities benefited from them, which in turn was predicted by children’s grade level and analogical reasoning ability. Our findings suggest that it depends on children’s prerequisites how effectively they are able or willing to engage in more cognitively demanding learning activities.

Mobile technologies offer new opportunities for encouraging self-regulated learning (SRL) in children. In this pre-registered study, we tested what kind of mobile intervention helps children maintain a regular study routine for vocabulary learning. Study behavior was measured objectively and with high ecological validity using logfiles of a vocabulary app that the children used. The mobile intervention was delivered via a separate study app and combined two critical components: planning and prompting. Children (N = 130, mean age = 10.75 years) first received a digital instruction on the benefits of distributed practice. Children in the full intervention group also formulated a plan for when and where to learn vocabulary. On about half of the next 36 days (within-subject manipulation), they received prompts reminding them of the instruction and the plan. The comparison groups lacked either the prompting or planning intervention component (between-person manipulation). The results revealed that the reminder prompts increased the likelihood of studying on the day they were given. Planning, in contrast, had a more long-term impact in that it helped children maintain a high frequency of learning over time. Overall, our findings show that mobile interventions can be highly effective in supporting SRL in children. We discuss the results in light of theoretical assumptions about the different mechanisms underlying planning and prompting effects, and highlight the practical importance of finding the optimal frequency of prompting.

App-based habit building has been shown to be a good tool for forming desired habits; however, it is unclear how much individual features that are present in many apps contribute to the success of habit building. In this paper, the authors consider the influence of social support features by developing an app in which habit progress was shared with peers – 'buddies' in the app. In the study, 38 participants created habits and monitored their progress regularly with the app over three weeks. The participants were divided into a control group without a 'buddy' and a treatment group cohort in which they were assigned to buddies based on their desired habits. With each habit repetition, the app gave feedback on the number of repetitions and the automaticity of the user's habit. The results obtained show that the reproduction of app-based intentional habit building is effective and that automaticity could be predicted by habit repetition.

When reading long and complex texts, students may disengage and miss out on relevant content. In order to prevent disengaged behavior or to counteract it by means of an intervention, it is ideally detected an early stage. In this paper, we present a method for early disengagement detection that relies only on the classification of scrolling data. The presented method transforms scrolling data into a time series representation, where each point of the series represents the vertical position of the viewport in the text document. This time series representation is then classified using time series classification algorithms. We evaluated the method on a dataset of 565 university students reading eight different texts. We compared the algorithm performance with different time series lengths, data sampling strategies, the texts that make up the training data, and classification algorithms. The method can classify disengagement early with up to 70% accuracy. However, we also observe differences in the performance depending on which of the texts are included in the training dataset. We discuss our results and propose several possible improvements to enhance the method.

Multimodal learning analytics can enrich interaction data with contextual information through mobile sensing. Information about, for example, the physical environment, movement, physiological signals, or smart wearable usage. Through the use of smart wearables, contextual information can thus be captured and made available again to students in further processing steps so that they can reflect and annotate it. This paper describes a software infrastructure and a study design that successfully captured contextual information utilizing mobile sensing using students’ smart wearables in distance learning. In the conducted study, data was collected from the smartphones of 76 students as they self-directedly participated in an online learning unit using a learning management system (LMS) over a two-week period. During the students’ active phases in the LMS, interaction data as well as state and trait measurements were collected by the LMS. Simultaneously, hardware sensor data, app usage data, interaction with notifications, and ecological momentary assessments (EMA) were automatically but transparently collected from the students’ smartphones. Finally, this paper describes some preliminary insights from the study process and their implications for further data processing.

Digital distractions can interfere with goal attainment and lead to undesirable habits that are hard to get red rid of. Various digital self-control interventions promise support to alleviate the negative impact of digital distractions. These interventions use different approaches, such as the blocking of apps and websites, goal setting, or visualizations of device usage statistics. While many apps and browser extensions make use of these features, little is known about their effectiveness. This systematic review synthesizes the current research to provide insights into the effectiveness of the different kinds of interventions. From a search of the 'ACM', 'Springer Link', 'Web of Science', 'IEEE Xplore' and 'Pubmed' databases, we identified 28 digital self-control interventions. We categorized these interventions according to their features and their outcomes. The interventions showed varying degrees of effectiveness, and especially interventions that relied purely on increasing the participants' awareness were barely effective. For those interventions that sanctioned the use of distractions, the current literature indicates that the sanctions have to be sufficiently difficult to overcome, as they will otherwise be quickly dismissed. The overall confidence in the results is low, with small sample sizes, short study duration, and unclear study contexts. From these insights, we highlight research gaps and close with suggestions for future research.

We report the results of a study that investigated the usage of a consumer-grade smartwatch for associative media learning. In a study with 29 participants, we investigated the effects of learning the Morse code alphabet purely visual versus learning with an associated vibration pattern. Our results show a trend that participants who received the vibration stimulus in addition to the visual stimulus learned faster and retained more knowledge than the group who learned using only the visualization, as measured by two post-tests one and three days after the learning took place.

The research described in this article covers the user-facing components of a learning analytics environment which is developed with the premise of trust as an essential factor for the adoption of learning analytics. It investigates the influence of privacy settings and personalization on the acceptance and adoption of learning analytics. By ensuring compliance with data protection legislation, and by providing transparency in the means and results of data collection, we aim to reduce doubts and fears in the learning analytics process. By respecting the needs of individuals, we hope to create an environment where learning analytics is perceived as something positive.

This paper presents the Learning Analytics Indicator Repository (LAIR), an interactive web-based application that allows the exploration of learning analytics approaches. From scientific publications in the field of learning analytics, we extracted the stakeholders, metrics, platforms and indicators, and transformed them into a directed graph representation. The LAIR allows filtering by these components and provides a list of publications where the approaches can be found. We invite other researchers to contribute to this repository.